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Chapter 1: Getting started with Python 
Language 


Python 3.x 
Version Release Date 
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25) 2006-09-19 
2 2004-11-30 


2.3 2003-07-29 
22 2001-12-21 
Al 2001-04-15 
2.0 2000-10-16 


Section 1.1: Getting Started 


Python is a widely used high-level programming language for general-purpose programming, created by Guido van 
Rossum and first released in 1991. Python features a dynamic type system and automatic memory management 
and supports multiple programming paradigms, including object-oriented, imperative, functional programming, 
and procedural styles. It has a large and comprehensive standard library. 


Two major versions of Python are currently in active use: 


e Python 3.x is the current version and is under active development. 
e Python 2.x is the legacy version and will receive only security updates until 2020. No new features will be 
implemented. Note that many projects still use Python 2, although migrating to Python 3 is getting easier. 


You can download and install either version of Python here. See Python 3 vs. Python 2 for a comparison between 
them. In addition, some third-parties offer re-packaged versions of Python that add commonly used libraries and 
other features to ease setup for common use cases, such as math, data analysis or scientific use. See the list at the 
official site. 


Verify if Python is installed 


To confirm that Python was installed correctly, you can verify that by running the following command in your 
favorite terminal (If you are using Windows OS, you need to add path of python to the environment variable before 
using it in command prompt): 
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S$ python --version 


Python 3.x Version = 3.0 


If you have Python 3 installed, and it is your default version (see Troubleshooting for more details) you should see 
something like this: 


S$ python --version 
Python 3.6.0 


Python 2.x Version < 2.7 


If you have Python 2 installed, and it is your default version (see Troubleshooting for more details) you should see 
something like this: 


S$ python --version 
Python 2.7.13 


If you have installed Python 3, but $ python --version outputs a Python 2 version, you also have Python 2 
installed. This is often the case on MacOS, and many Linux distributions. Use $ python3 instead to explicitly use the 
Python 3 interpreter. 


Hello, World in Python using IDLE 
IDLE is a simple editor for Python, that comes bundled with Python. 
How to create Hello, World program in IDLE 


e Open IDLE on your system of choice. 
o In older versions of Windows, it can be found at All Programs under the Windows menu. 
° In Windows 8+, search for IDLE or find it in the apps that are present in your system. 
° On Unix-based (including Mac) systems you can open it from the shell by typing $ idle 
python_file.py. 
e It will open a shell with options along the top. 


In the shell, there is a prompt of three right angle brackets: 


Now write the following code in the prompt: 


>>> print("Hello, World") 


Hit|_ Enter | 


>>> print("Hello, World") 
Hello, World 


Hello World Python file 
Create a new file hello. py that contains the following line: 


Python 3.x Version = 3.0 
print('Hello, World’) 


Python 2.x Version = 2.6 


Goalkicker.com - Python® Notes for Professionals 3 


You can use the Python 3 print function in Python 2 with the following import statement: 


from __future__ import print_function 


Python 2 has a number of functionalities that can be optionally imported from Python 3 using the __future__ 
module, as discussed here. 


Python 2.x Version < 2.7 


If using Python 2, you may also type the line below. Note that this is not valid in Python 3 and thus not 
recommended because it reduces cross-version code compatibility. 


print ‘Hello, World' 


In your terminal, navigate to the directory containing the file hello. py. 


Type python hello.py, then hit the key. 


S$ python hello. py 
Hello, World 


You should see Hello, World printed to the console. 


You can also substitute hello. py with the path to your file. For example, if you have the file in your home directory 
and your user is "user" on Linux, you can type python /home/user/hello.py. 


Launch an interactive Python shell 


By executing (running) the python command in your terminal, you are presented with an interactive Python shell. 
This is also known as the Python Interpreter or a REPL (for 'Read Evaluate Print Loop’). 


S$ python 

Python 2.7.12 (default, Jun 28 2016, 08:46:01) 

[GCC 6.1.1 280160602] on linux 

Type "help", "copyright", "credits" or "license" for more information. 
>>> print ‘Hello, World' 

Hello, World 


>>> 


If you want to run Python 3 from your terminal, execute the command python3. 


S$ python3 

Python 3.6.0 (default, Jan 13 2017, 00:00:00) 

[GCC 6.1.1 28160602] on linux 

Type "help", "copyright", "credits" or "license" for more information. 
>>> print('Hello, World') 

Hello, World 


>>> 
Alternatively, start the interactive prompt and load file with python -i <file.py>. 
In command line, run: 


S$ python -i hello.py 
"Hello World" 


>>> 
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There are multiple ways to close the Python shell: 

>>> exit() 

or 

>>> quit() 

Alternatively, will close the shell and put you back on your terminal's command line. 


If you want to cancel a command you're in the middle of typing and get back to a clean command prompt, while 


staying inside the Interpreter shell, use| CTRL + C |. 


Try an interactive Python shell online. 


Other Online Shells 
Various websites provide online access to Python shells. 
Online shells may be useful for the following purposes: 


e Runasmall code snippet from a machine which lacks python installation(smartphones, tablets etc). 
e Learn or teach basic Python. 
e Solve online judge problems. 


Examples: 


Disclaimer: documentation author(s) are not affiliated with any resources listed below. 


https://www.python.org/shell/ - The online Python shell hosted by the official Python website. 
https://ideone.com/ - Widely used on the Net to illustrate code snippet behavior. 
https://repl|.it/languages/python3 - Powerful and simple online compiler, IDE and interpreter. Code, compile, 
and run code in Python. 

https://www.tutorialspoint.com/execute python online.php - Full-featured UNIX shell, and a user-friendly 
project explorer. 

http://rextester.com/I/python3 online compiler - Simple and easy to use IDE which shows execution time 


Run commands as a string 


Python can be passed arbitrary code as a string in the shell: 


$ python -c 'print("Hello, World") 
Hello, World 


This can be useful when concatenating the results of scripts together in the shell. 
Shells and Beyond 


Package Management - The PyPA recommended tool for installing Python packages is PIP. To install, on your 
command line execute pip install <the package name>. For instance, pip install numpy. (Note: On windows 
you must add pip to your PATH environment variables. To avoid this, use python -m pip install <the package 
name>) 


Shells - So far, we have discussed different ways to run code using Python's native interactive shell. Shells use 
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Python's interpretive power for experimenting with code real-time. Alternative shells include IDLE - a pre-bundled 
GUI, [Python - known for extending the interactive experience, etc. 


Programs - For long-term storage you can save content to .py files and edit/execute them as scripts or programs 
with external tools e.g. shell, IDEs (such as PyCharm), Jupyter notebooks, etc. Intermediate users may use these 
tools; however, the methods discussed here are sufficient for getting started. 


Python tutor allows you to step through Python code so you can visualize how the program will flow, and helps you 
to understand where your program went wrong. 


PEP8 defines guidelines for formatting Python code. Formatting code well is important so you can quickly read what 
the code does. 


Section 1.2: Creating variables and assigning values 
To create a variable in Python, all you need to do is specify the variable name, and then assign a value to it. 


<variable name> = <value> 


Python uses = to assign values to variables. There's no need to declare a variable in advance (or to assign a data 
type to it), assigning a value to a variable itself declares and initializes the variable with that value. There's no way to 
declare a variable without assigning it an initial value. 


# Integer 
a=2 
print(a) 

# Output: 2 


# Integer 

b = 9223372036854775807 
print(b) 

# Output: 9223372036854775807 


# Floating point 
pi = 3.14 
print(pi) 

# Output: 3.14 


# String 
c= 'A' 
print(c) 

# Output: A 


# String 

name = ‘John Doe' 
print (name) 

# Output: John Doe 


# Boolean 
q = True 


print(q) 
# Output: True 


# Empty value or null data type 
x = None 

print(x) 

# Output: None 
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Variable assignment works from left to right. So the following will give you an syntax error. 


@ =x 
=> Output: SyntaxError: can't assign to literal 


You can not use python's keywords as a valid variable name. You can see the list of keyword by: 


import keyword 
print (keyword.kwlist) 


Rules for variable naming: 
1. Variables names must start with a letter or an underscore. 


True # valid 
_y = True’ # valid 


x 
I 


9x = False # starts with numeral 
=> SyntaxError: invalid syntax 


Sy = False # starts with symbol 
=> SyntaxError: invalid syntax 


2. The remainder of your variable name may consist of letters, numbers and underscores. 
has_@_in_it = "Still Valid" 
3. Names are case sensitive. 


x = 9 
y = X*5 
=>NameError: name 'X' is not defined 


Even though there's no need to specify a data type when declaring a variable in Python, while allocating the 
necessary area in memory for the variable, the Python interpreter automatically picks the most suitable built-in 
type for it: 


a=2 


print(type(a) ) 
# Output: <type ‘int'> 


b = 9223372036854775807 


print(type(b) ) 
# Output: <type ‘int'> 


pi = 3.14 


print (type(pi)) 
# Output: <type ‘float'> 


@ Soy\ 


print(type(c)) 
# Output: <type ‘str'> 


name = ‘John Doe' 
print (type(name) ) 
# Output: <type 'str'> 


q = True 
print(type(q) ) 
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# Output: <type 'bool'> 


x = None 
print(type(x) ) 
# Output: <type 'NoneType'> 


Now you know the basics of assignment, let's get this subtlety about assignment in python out of the way. 


When you use = to do an assignment operation, what's on the left of = is a name for the object on the right. Finally, 
what = does is assign the reference of the object on the right to the name on the left. 


That is: 
a_name = an_object # "a_name" is now a name for the reference to the object "an_object” 


So, from many assignment examples above, if we pick pi = 3.14, then pi is a name (not the name, since an object 
can have multiple names) for the object 3.14. If you don't understand something below, come back to this point 
and read this again! Also, you can take a look at this for a better understanding. 


You can assign multiple values to multiple variables in one line. Note that there must be the same number of 
arguments on the right and left sides of the = operator: 


abr Cu== (re agen 
print(a, b, c) 
# Output: 71 2 3 


atm DeeCe— ele 

=> Traceback (most recent call last): 

=> File "name.py", line N, in <module> 

=> Ely 10 Os Sg 

=> ValueError: need more than 2 values to unpack 


a, b> 4, 2, 3 

=> Traceback (most recent call last): 

=> File "name.py", line N, in <module> 
=> ae bias 1 2S 

=> ValueError: too many values to unpack 


The error in last example can be obviated by assigning remaining values to equal number of arbitrary variables. 
This dummy variable can have any name, but it is conventional to use the underscore (_) for assigning unwanted 
values: 


ei in ee le a 


print(a, b) 
# Output: 71, 2 


Note that the number of _ and number of remaining values must be equal. Otherwise 'too many values to unpack 
error’ is thrown as above: 


Elen (ope ent all pe anesuae 
=>Traceback (most recent call last): 
=>File "name.py", line N, in <module> 
=>a, b, = 152,34 


=>ValueError: too many values to unpack (expected 3) 


You can also assign a single value to several variables simultaneously. 
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a=b=ce=t1 
print(a, b, c) 
# Output: 17 7 


When using such cascading assignment, it is important to note that all three variables a, b and c refer to the same 
object in memory, an int object with the value of 1. In other words, a, b and c are three different names given to the 
same int object. Assigning a different object to one of them afterwards doesn't change the others, just as expected: 


a=b=ce=1 # all three names a, b and c refer to same int object with value 1 


print(a, b, c) 


# Output: 71 7 7 
be=eZ #b now refers to another int object, one with a value of 2 


print(a, b, c) 
# Output: 12 1 # so output is as expected. 


The above is also true for mutable types (like list, dict, etc.) just as it is true for immutable types (like int, string, 
tuple, etc.): 
saws (iv, 2 Sil # x and y refer to the same list object just created, [7, 8, 9] 


x = [13, 8, 9] # x now refers to a different list object just created, [13, 8, 9] 
print(y) # y still refers to the list it was first assigned 


# Output: [7, 8, 9] 


So far so good. Things are a bit different when it comes to modifying the object (in contrast to assigning the name to 
a different object, which we did above) when the cascading assignment is used for mutable types. Take a look 


below, and you will see it first hand: 


See WS (Pe he el # x and y are two different names for the same list object just created, [7, 
8, 9] 

x[@] = 13 # we are updating the value of the list [7, 8, 9] through one of its names, x 
in this case 

print(y) # printing the value of the list using its other name 


# Output: [13, 8, 9] # hence, naturally the change is reflected 
Nested lists are also valid in python. This means that a list can contain another list as an element. 


x= 2, [8s 4.5 6, 714 thisias nested list 
print x[2] 

AS OUGE DU ano rea.) 

print x[2][1] 

# Output: 4 


Lastly, variables in Python do not have to stay the same type as which they were first defined -- you can simply use 
= to assign a new value to a variable, even if that value is of a different type. 


a=2 
print(a) 
# Output: 2 


a = "New value" 
print(a) 
# Output: New value 


If this bothers you, think about the fact that what's on the left of = is just a name for an object. First you call the int 
object with value 2 a, then you change your mind and decide to give the name a to a string object, having value 


‘New value’. Simple, right? 
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Section 1.3: Block Indentation 


Python uses indentation to define control and loop constructs. This contributes to Python's readability, however, it 
requires the programmer to pay close attention to the use of whitespace. Thus, editor miscalibration could result in 
code that behaves in unexpected ways. 


Python uses the colon symbol (:) and indentation for showing where blocks of code begin and end (If you come 
from another language, do not confuse this with somehow being related to the ternary operator). That is, blocks in 
Python, such as functions, loops, if clauses and other constructs, have no ending identifiers. All blocks start with a 
colon and then contain the indented lines below it. 


For example: 


def my_function(): # This is a function definition. Note the colon (:) 
a=2 # This line belongs to the function because it's indented 
return a # This line also belongs to the same function 
print(my_function()) # This line is OUTSIDE the function block 
or 
if a>b: # If block starts here 
print(a) # This is part of the if block 
else: # else must be at the same level as if 
print(b) # This line is part of the else block 


Blocks that contain exactly one single-line statement may be put on the same line, though this form is generally not 
considered good style: 


if a > b: print(a) 
else: print(b) 


Attempting to do this with more than a single statement will not work: 


if x >y: y =x 
print(y) # IndentationError: unexpected indent 


if x > y: while y != z: y -= 1. # SyntaxError: invalid syntax 


An empty block causes an IndentationError. Use pass (a command that does nothing) when you have a block with 
no content: 


def will_be_implemented_later(): 
pass 


Spaces vs. Tabs 

In short: always use 4 spaces for indentation. 

Using tabs exclusively is possible but PEP 8, the style guide for Python code, states that spaces are preferred. 
Python 3.x Version = 3.0 


Python 3 disallows mixing the use of tabs and spaces for indentation. In such case a compile-time error is 
generated: Inconsistent use of tabs and spaces in indentation and the program will not run. 


Python 2.x Version < 2.7 
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Python 2 allows mixing tabs and spaces in indentation; this is strongly discouraged. The tab character completes 
the previous indentation to be a multiple of 8 spaces. Since it is common that editors are configured to show tabs 


as multiple of 4 spaces, this can cause subtle bugs. 


Citing PEP 8: 


When invoking the Python 2 command line interpreter with the -t option, it issues warnings about code 
that illegally mixes tabs and spaces. When using -tt these warnings become errors. These options are 
highly recommended! 


Many editors have "tabs to spaces" configuration. When configuring the editor, one should differentiate between 


the tab character (‘\t') and the key. 


e The tab character should be configured to show 8 spaces, to match the language semantics - at least in cases 
when (accidental) mixed indentation is possible. Editors can also automatically convert the tab character to 
spaces. 

e However, it might be helpful to configure the editor so that pressing the key will insert 4 spaces, 
instead of inserting a tab character. 


Python source code written with a mix of tabs and spaces, or with non-standard number of indentation spaces can 
be made pep8-conformant using autopep8. (A less powerful alternative comes with most Python installations: 


reindent.py) 


Section 1.4: Datatypes 


Built-in Types 
Booleans 


bool: A boolean value of either True or False. Logical operations like and, or, not can be performed on booleans. 


x or y # if x is False then y otherwise x 
x and y # if x is False then x otherwise y 
not x # if x is True then False, otherwise True 


In Python 2.x and in Python 3.x, a boolean is also an int. The bool type is a subclass of the int type and True and 
False are its only instances: 


issubclass(bool, int) # True 


isinstance(True, bool) # True 
isinstance(False, bool) # True 


If boolean values are used in arithmetic operations, their integer values (1 and 8 for True and False) will be used to 
return an integer result: 


True + False == 1 
True * True == 1 


Numbers 


e int: Integer number 
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2 

100 

123456789 

= 38563846326424324 


a0°0 © 


Integers in Python are of arbitrary sizes. 


Note: in older versions of Python, a long type was available and this was distinct from int. The two have 
been unified. 


¢ float: Floating point number; precision depends on the implementation and system architecture, for 
CPython the float datatype corresponds to a C double. 


250 
100.e0 
c = 123456789.e1 


ao ® 
oll 


¢ complex: Complex numbers 


a=2+1j 
100 + 10; 


ion 
I 


The <, <=, > and >= operators will raise a TypeError exception when any operand is a complex number. 


Strings 
Python 3.x Version = 3.0 


e str: a unicode string. The type of ‘hello’ 
° bytes: a byte string. The type of b'hello' 


Python 2.x Version < 2.7 


e str: a byte string. The type of ‘hello’ 
e bytes: synonym for str 
¢ unicode: a unicode string. The type of u'hello' 


Sequences and collections 
Python differentiates between ordered sequences and unordered collections (such as set and dict). 
e strings (str, bytes, unicode) are sequences 
e reversed: Areversed order of str with reversed function 
a = reversed('hello' ) 


e tuple: An ordered collection of n values of any type (n >= 8). 


cle Gye’s ec) 
be= (Gand apychomis walle 2)s) 
b[2] = ‘something else' # returns a TypeError 


Supports indexing; immutable; hashable if all its members are hashable 
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e list: An ordered collection of n values (n >= 8) 


ea | 
Se, “esas (il, Ce, [il lll 
[2] = ‘something else' # allowed 


Not hashable; mutable. 


e set: An unordered collection of unique values. Items must be hashable. 


2: '‘two'} 


baa! [V5 3); 
beets a Strange} 


An object is hashable if it has a hash value which never changes during its lifetime (it needs a ___hash__() 
method), and can be compared to other objects (it needs an __eq__() method). Hashable objects which 
compare equality must have the same hash value. 


Built-in constants 


In conjunction with the built-in datatypes there are a small number of built-in constants in the built-in namespace: 


True: The true value of the built-in type bool 

False: The false value of the built-in type bool 

None: A singleton object used to signal that a value is absent. 

Ellipsis or ...: used in core Python3+ anywhere and limited usage in Python2.7+ as part of array notation. 
numpy and related packages use this as a ‘include everything’ reference in arrays. 

NotImplemented: a singleton used to indicate to Python that a special method doesn't support the specific 
arguments, and Python will try alternatives if available. 


a = None # No value will be assigned. Any valid datatype can be assigned later 


Python 3.x Version = 3.0 


None doesn't have any natural ordering. Using ordering comparison operators (<, <=, >=, >) isn't supported anymore 
and will raise a TypeError. 


Python 2.x Version < 2.7 
None is always less than any number (None < -32 evaluates to True). 
Testing the type of variables 


In python, we can check the datatype of an object using the built-in function type. 


ava) 23) 
print(type(a) ) 
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# Out: <class ‘str'> 
bear 23 
print(type(b) ) 

# Out: <class ‘int'> 


In conditional statements it is possible to test the datatype with isinstance. However, it is usually not encouraged 


to rely on the type of the variable. 


i=7 
if isinstance(i, int): 
its 1 


elif isinstance(i, str): 
i = int(i) 
a 


For information on the differences between type() and isinstance() read: Differences between isinstance and 


type in Python 


To test if something is of NoneType: 
x = None 


if x is None: 
print('Not a surprise, I just defined x as None.') 


Converting between datatypes 
You can perform explicit datatype conversion. 
For example, '123' is of str type and it can be converted to integer using int function. 


= '123' 
b = int(a) 


o 
I 


Converting from a float string such as '123.456' can be done using float function. 


= '123.456' 

= float(a) 

= int(a) # ValueError: invalid literal for int() with base 1@: '123.456' 
= int(b) # 123 


a00 © 


You can also convert sequence or collection types 


a = ‘hello' 

IIGIERACED) Ae Utne BG SAT Il oy | 
set(a) pe eM oy ts LY S eTah 
tuple(ajee (he wes. al a eos) 


Explicit string type at definition of literals 
With one letter labels just in front of the quotes you can tell what type of string you want to define. 


¢ b' foo bar': results bytes in Python 3, str in Python 2 

e u' foo bar’: results str in Python 3, unicode in Python 2 

e ‘foo bar': results str 

e r' foo bar': results so called raw string, where escaping special characters is not necessary, everything is 
taken verbatim as you typed 


normal = ‘foo\nbar' # foo 
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# bar 
escaped = 'foo\\nbar' # foo\nbar 
raw r'foo\nbar' # foo\nbar 


Mutable and Immutable Data Types 


An object is called mutable if it can be changed. For example, when you pass a list to some function, the list can be 
changed: 


def f(m): 
m.append(3) # adds a number to the list. This is a mutation. 
x= (1, 2] 
F(x) 
x == [1, 2] # False now, since an item was added to the list 


An object is called immutable if it cannot be changed in any way. For example, integers are immutable, since there's 
no way to change them: 


def bar(): 
4S (Gh 2) 
g(x) 
x == (1, 2) # Will always be True, since no function can change the object (1, 2) 


Note that variables themselves are mutable, so we can reassign the variable x, but this does not change the object 
that x had previously pointed to. It only made x point to a new object. 


Data types whose instances are mutable are called mutable data types, and similarly for immutable objects and 
datatypes. 


Examples of immutable Data Types: 


e int, long, float, complex 
e str 

e bytes 

e tuple 

e frozenset 


Examples of mutable Data Types: 


e bytearray 
e list 

e set 

e dict 


Section 1.5: Collection Types 


There are a number of collection types in Python. While types such as int and str hold a single value, collection 
types hold multiple values. 


Lists 


The list type is probably the most commonly used collection type in Python. Despite its name, a list is more like an 
array in other languages, mostly JavaScript. In Python, a list is merely an ordered collection of valid Python values. A 
list can be created by enclosing values, separated by commas, in square brackets: 
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intolas tea [dee si] 
string_list = ['abc', ‘defghi'] 


A list can be empty: 
empty_list = [] 


The elements of a list are not restricted to a single data type, which makes sense given that Python is a dynamic 
language: 

mixed_list = [1, ‘abc', True, 2.34, None] 

A list can contain another list as its element: 

(oiled alles ee 2 3 || 


nested_list = [[‘a', ‘b', 


The elements of a list can be accessed via an index, or numeric representation of their position. Lists in Python are 
zero-indexed meaning that the first element in the list is at index 0, the second element is at index 1 and so on: 


names = ['Alice', 'Bob', ‘Craig', ‘'Diana', ‘Eric'] 
print(names[@]) # Alice 
print(names[2]) # Craig 


Indices can also be negative which means counting from the end of the list (-1 being the index of the last element). 
So, using the list from the above example: 


print(names[-1]) # Eric 
print(names[-4]) # Bob 


Lists are mutable, so you can change the values in a list: 


names[@] = ‘Ann' 
print (names) 
# Outputs ['Ann', 'Bob', ‘Craig’, ‘Diana', ‘Eric’ ] 


Besides, it is possible to add and/or remove elements from a list: 

Append object to end of list with L.append(object), returns None. 
names = ['Alice', 'Bob', ‘Craig', ‘Diana', ‘Eric'] 
names.append( "Sia" ) 


print (names) 
# Outputs ['Alice', '‘Bob', ‘Craig', ‘Diana’, ‘Eric’, '‘Sia'] 


Add a new element to list at a specific index. L.insert(index, object) 


names.insert(1, "Nikki") 
print (names) 
# Outputs ['Alice', 'Nikki', '‘Bob', ‘Craig’, ‘Diana', ‘Eric', 'Sia'] 


Remove the first occurrence of a value with L. remove(value), returns None 
names. remove( "Bob" ) 


print(names) # Outputs ['Alice', ‘Nikki’, ‘Craig', ‘Diana', ‘Eric', ‘Sia'] 
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Get the index in the list of the first item whose value is x. It will show an error if there is no such item. 


name. index("Alice" ) 
) 


Count length of list 


len(names) 
6 


count occurrence of any item in list 
a = [1 I 1 I 1 , 2, 35 4] 


a.count(1) 
3 


Reverse the list 


a.reverse() 

[4, 3) Z 1 , iy 1 ] 

# or 

a[::-1] 

[4, 3 2s 1 , 1; 1 ] 

Remove and return item at index (defaults to the last item) with L. pop( [index]), returns the item 
names.pop() # Outputs ‘Sia’ 


You can iterate over the list elements like below: 


for element in my_list: 
print (element) 


Tuples 


A tuple is similar to a list except that it is fixed-length and immutable. So the values in the tuple cannot be changed 
nor the values be added to or removed from the tuple. Tuples are commonly used for small collections of values 
that will not need to change, such as an IP address and port. Tuples are represented with parentheses instead of 
square brackets: 


ip_address = ('10.20.30.40', 8080) 


The same indexing rules for lists also apply to tuples. Tuples can also be nested and the values can be any valid 
Python valid. 


A tuple with only one member must be defined (note the comma) this way: 
one_member_tuple = (‘Only member’, ) 


or 


one_member_tuple = ‘Only member', # No brackets 


or just using tuple syntax 
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one_member_tuple = tuple(['Only member’ ]) 


Dictionaries 


A dictionary in Python is a collection of key-value pairs. The dictionary is surrounded by curly braces. Each pair is 
separated by a comma and the key and value are separated by a colon. Here is an example: 


state_capitals = { 
"Arkansas': ‘Little Rock', 


‘Colorado': ‘Denver’, 
‘California’: 'Sacramento', 
‘Georgia’: ‘Atlanta’ 


To get a value, refer to it by its key: 
ca_capital = state_capitals['California' ] 
You can also get all of the keys in a dictionary and then iterate over them: 


for k in state_capitals.keys(): 
print('{} is the capital of {}'.format(state_capitals[k], k)) 


Dictionaries strongly resemble JSON syntax. The native json module in the Python standard library can be used to 
convert between JSON and dictionaries. 


set 


A set is a collection of elements with no repeats and without insertion order but sorted order. They are used in 
situations where it is only important that some things are grouped together, and not what order they were 
included. For large groups of data, it is much faster to check whether or not an element is in a set than it is to do 
the same for a list. 


Defining a set is very similar to defining a dictionary: 
first_names = {'Adam', 'Beth', ‘Charlie’ } 
Or you can build a set using an existing list: 


my_list = [1,2,3] 
my_set = set(my_list) 


Check membership of the set using in: 


if name in first_names: 
print (name) 


You can iterate over a set exactly like a list, but remember: the values will be in an arbitrary, implementation- 
defined order. 


defaultdict 


A defaultdict is a dictionary with a default value for keys, so that keys for which no value has been explicitly 
defined can be accessed without errors. defaultdict is especially useful when the values in the dictionary are 
collections (lists, dicts, etc) in the sense that it does not need to be initialized every time when a new key is used. 
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A defaultdict will never raise a KeyError. Any key that does not exist gets the default value returned. 
For example, consider the following dictionary 


>>> state_capitals = { 
"Arkansas': ‘Little Rock', 
‘Colorado': '‘Denver', 
‘California’: 'Sacramento', 
‘Georgia’: ‘Atlanta’ 


If we try to access a non-existent key, python returns us an error as follows 


>>> state_capitals[ ‘Alabama’ ] 
Traceback (most recent call last): 


File "<ipython-input-61-236329695e6f>", line 1, in <module> 
state_capitals[ ‘Alabama’ ] 


KeyError: ‘Alabama’ 
Let us try with a defaultdict. It can be found in the collections module. 


>>> from collections import defaultdict 
>>> state_capitals = defaultdict(lambda: ‘Boston’ ) 


What we did here is to set a default value (Boston) in case the give key does not exist. Now populate the dict as 
before: 


>>> state_capitals[ ‘Arkansas’ ] ‘Little Rock’ 


>>> state_capitals['California'] = ‘Sacramento’ 
>>> state_capitals['Colorado'] = ‘Denver’ 
>>> state_capitals['Georgia'] = ‘Atlanta’ 


If we try to access the dict with a non-existent key, python will return us the default value i.e. Boston 


>>> state_capitals[ ‘Alabama’ ] 
‘Boston’ 


and returns the created values for existing key just like a normal dictionary 


>>> state_capitals[ ‘Arkansas’ ] 
"Little Rock' 


Section 1.6: IDLE - Python GUI 


IDLE is Python’s Integrated Development and Learning Environment and is an alternative to the command line. As 
the name may imply, IDLE is very useful for developing new code or learning python. On Windows this comes with 
the Python interpreter, but in other operating systems you may need to install it through your package manager. 


The main purposes of IDLE are: 


e Multi-window text editor with syntax highlighting, autocompletion, and smart indent 
e Python shell with syntax highlighting 

e Integrated debugger with stepping, persistent breakpoints, and call stack visibility 

e Automatic indentation (useful for beginners learning about Python's indentation) 
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Saving the Python program as .py files and run them and edit them later at any them using IDLE. 


In IDLE, hit F5 or run Python Shell to launch an interpreter. Using IDLE can be a better learning experience for 
new users because code is interpreted as the user writes. 


Note that there are lots of alternatives, see for example this discussion or this list. 


Troubleshooting 


Windows 


If you're on Windows, the default command is python. If you receive a""'python' is not recognized" error, 
the most likely cause is that Python's location is not in your system's PATH environment variable. This can be 
accessed by right-clicking on 'My Computer’ and selecting 'Properties' or by navigating to 'System' through 
‘Control Panel’. Click on ‘Advanced system settings' and then ‘Environment Variables...'. Edit the PATH variable 
to include the directory of your Python installation, as well as the Script folder (usually 

C:\Python27 ;C:\Python27\Scripts). This requires administrative privileges and may require a restart. 


When using multiple versions of Python on the same machine, a possible solution is to rename one of the 
python .exe files. For example, naming one version python27.exe would cause python27 to become the 
Python command for that version. 


You can also use the Python Launcher for Windows, which is available through the installer and comes by 
default. It allows you to select the version of Python to run by using py -[x.y] instead of python[x.y]. You 
can use the latest version of Python 2 by running scripts with py -2 and the latest version of Python 3 by 
running scripts with py -3. 


Debian/Ubuntu/MacOS 


This section assumes that the location of the python executable has been added to the PATH environment 
variable. 


If you're on Debian/Ubuntu/MacOS, open the terminal and type python for Python 2.x or python3 for Python 
3.x. 


Type which python to see which Python interpreter will be used. 


Arch Linux 


The default Python on Arch Linux (and descendants) is Python 3, so use python or python3 for Python 3.x and 
python2 for Python 2.x. 


Other systems 


Python 3 is sometimes bound to python instead of python3. To use Python 2 on these systems where it is 
installed, you can use python2. 
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Section 1.7: User Input 


Interactive input 


To get input from the user, use the input function (note: in Python 2.x, the function is called raw_input instead, 
although Python 2.x has its own version of input that is completely different): 


Python 2.x Version = 2.3 


name = raw_input("What is your name? ") 
# Out: What is your name? _ 


Security Remark Do not use input() in Python2 - the entered text will be evaluated as if it were a 
Python expression (equivalent to eval(input()) in Python3), which might easily become a vulnerability. 
See this article for further information on the risks of using this function. 


Python 3.x Version = 3.0 


name = input("What is your name? ") 
# Out: What is your name? _ 


The remainder of this example will be using Python 3 syntax. 


The function takes a string argument, which displays it as a prompt and returns a string. The above code provides a 
prompt, waiting for the user to input. 


name = input("What is your name? ") 
# Out: What is your name? 


If the user types "Bob" and hits enter, the variable name will be assigned to the string "Bob": 


name = input("What is your name? ") 
# Out: What is your name? Bob 

print (name) 

# Out: Bob 


Note that the input is always of type str, which is important if you want the user to enter numbers. Therefore, you 
need to convert the str before trying to use it as a number: 


x = input("Write a number:") 

# Out: Write a number: 10 

Xa eee 

# Out: TypeError: unsupported operand type(s) for /: ‘str’ and ‘int' 
float(x) / 2 

# Out: 5.0 


NB: It's recommended to use try/except blocks to catch exceptions when dealing with user inputs. For instance, if 
your code wants to cast a raw_input into an int, and what the user writes is uncastable, it raises a ValueError. 


Section 1.8: Built in Modules and Functions 


A module is a file containing Python definitions and statements. Function is a piece of code which execute some 
logic. 


>>> pow(2,3) #8 
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To check the built in function in python we can use dir(). If called without an argument, return the names in the 
current scope. Else, return an alphabetized list of names comprising (some of) the attribute of the given object, and 
of attributes reachable from it. 


>>> dir(__builtins__) 


‘ArithmeticError', 
"AssertionError', 
‘AttributeError', 
"BaseException', 
'BufferError', 
"BytesWarning' , 
‘DeprecationWarning' , 
JEOFERROn =: 

“Ellipsis”; 
‘EnvironmentError', 
‘Exception’, 

‘False’, 
"FloatingPointError'’, 
"FutureWarning', 
‘GeneratorExit', 
TOETKOR ; 

‘ImportError’, 
‘ImportWarning', 
‘IndentationError', 
‘IndexError', 

‘KeyError', 
‘Keyboardinterrupt' , 
"LookupError'’ , 
‘"MemoryError’, 
"NameError', 

"None', 

"NotImplemented', 
"NotImplementedError'’ , 
‘OSError’, 
‘OverflowError'’, 
"PendingDeprecationWarning' , 
‘ReferenceError', 
‘"RuntimeError', 
"RuntimeWarning', 
‘StandardError', 
"StopIteration', 
“SyntaxERronr 5 
"SyntaxWarning' 
"SystemError’, 
"SystemExit', 
‘TabError', 
‘True’, 
‘TypeError', 
"UnboundLocalError', 
"UnicodeDecodeError', 
"UnicodeEncodeError', 
"UnicodeError', 
"UnicodeTranslateError', 
"UnicodeWarning', 
'UserWarning’ , 
'ValueError'’, 
‘Warning’, 
‘ZeroDivisionError', 
'__debug__', 

paadocam: 
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eee POlitemeee 
-=_names=.. 
"__package 
‘abs', 
valli 
‘any’, 
‘apply’, 
‘basestring', 
eloaltale . 
‘bool’, 
‘buheRs 
‘bytearray', 
‘bytes’, 
‘callable’, 
eChittes 
‘classmethod' , 
‘cmp’, 
‘coerce’, 
‘compile’, 
‘complex’, 
‘copyright’ , 
‘credits: ; 
‘delattr', 
dict: ; 
oblige? 
‘divmod', 
‘enumerate’, 
‘eval', 
‘execfile', 
‘exit', 
“hile: 
hiolter 
hloak «. 
“format, 
‘frozenset', 
‘getattr', 
‘globals', 
‘hasattr', 
‘hash', 
‘help’, 
"hex', 

sirdige 
“input; 
Polini, 
‘intern’, 
‘isinstance’', 
‘issubclass', 
‘iter’, 
‘len', 
‘license’, 
Sister 
“locals. 
“Long, ; 
‘map’, 

BIMaxXie 
‘memoryview' , 
eMmainice, 
7NeXitas. 
‘object’, 
wOCusr 
‘open', 
Olndiee 
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pow’, 
soleeting 4 
‘property’, 
quite; 
‘range’, 
raw_input', 
reduce', 
reload', 
repr’, 
reversed’, 
round’, 


‘setattr', 
‘slice’, 
‘sorted’, 
“staticmethod', 
Sitar 
"sum', 
‘super’, 
“ctuUpLe:; 
‘type’, 
Sunaichires 
‘unicode’, 
ValeSi, 
"xrange', 
zip 


To know the functionality of any function, we can use built in function help . 


>>> help(max) 
Help on built-in function max in module __builtin__: 


max(...) 
max(iterable[, key=func]) -> value 
max(a, b, c, ...[, key=func]) -> value 


With a single iterable argument, return its largest item. 
With two or more arguments, return the largest argument. 


Built in modules contains extra functionalities. For example to get square root of a number we need to include math 


module. 


>>> import math 
>>> math.sqrt(16) # 4.8 


To know all the functions in a module we can assign the functions list to a variable, and then print the variable. 


>>> import math 
>>> dir(math) 


[-S=docls))  =!name=-.  22package=—" «, “acos: , <acoshi. 

‘asin', ‘asinh', ‘atan', ‘atan2', ‘atanh', ‘ceil’, ‘copysign', 
“cos, ‘cosh, “degrees, “ev, “eri ;, “erfics;, “exp, <expmily; 
‘fabs, factorial =“ filoon , “tmod:, “frexp). ~fsume. “gamma. 
‘hypot', ‘isinf', ‘isnan', '‘ldexp', ‘lgamma', ‘log', 'log1@', 
slogip: modi. {pies “pow! >) radians... sini _sinhi eSQrte. 
“tan. otanh’. “trune | 


it seems __doc__ is useful to provide some documentation in, say, functions 
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>>> math.__doc__ 
‘This module is always available. It provides access to the\nmathematical 


functions defined by the C standard.’ 


In addition to functions, documentation can also be provided in modules. So, if you have a file named 


helloWor1d. py like this: 


"""This is the module docstring. 


def sayHello(): 
"""This is the function docstring. 
return ‘Hello World' 


You can access its docstrings like this: 


>>> import helloWorld 

>>> helloWorld.__doc__ 

‘This is the module docstring.' 
>>> helloWorld.sayHello.__doc__ 
‘This is the function docstring.' 


e For any user defined type, its attributes, its class's attributes, and recursively the attributes of its class's base 


classes can be retrieved using dir() 


>>> class MyClassObject(object): 
pass 


>>> dir(MyClassObject) 


[peaclassmay sy sade atti w adi Clan. l= d0Cl™ == fOhlateme a sGetdtticlbutem. su salhashmnu: 
anit) = anodulle ==) newa = = neduce= | heduceZexa Ss neph-— =)» =setatthe.., 
"—-Sizeof__’, '_ stro’, “ -subcllasshook__', °__weakref__’] 


Any data type can be simply converted to string using a builtin function called str. This function is called by default 
when a data type is passed to print 


>>> str(123) # "123" 


Section 1.9: Creating a module 


A module is an importable file containing definitions and statements. 
A module can be created by creating a . py file. 


# hello.py 
def say_hello(): 
print("Hello!") 


Functions in a module can be used by importing the module. 


For modules that you have made, they will need to be in the same directory as the file that you are importing them 
into. (However, you can also put them into the Python lib directory with the pre-included modules, but should be 


avoided if possible.) 
S$ python 


>>> import hello 
>>> hello.say_hello() 
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=> "Hello!" 


Modules can be imported by other modules. 
# greet.py 


import hello 
hello.say_hello() 


Specific functions of a module can be imported. 


# greet.py 
from hello import say_hello 
say_hello() 


Modules can be aliased. 


# greet.py 
import hello as ai 
ai.say_hello() 


A module can be stand-alone runnable script. 
# run_hello.py 


if __name__ == '__main__': 


from hello import say_hello 
say_hello() 


Run it! 


S$ python run_hello.py 
=> sHelliol 


If the module is inside a directory and needs to be detected by python, the directory should contain a file named 
Sat alee Sioa 


Section 1.10: Installation of Python 2.7.x and 3.x 


Note: Following instructions are written for Python 2.7 (unless specified): instructions for Python 3.x are 
similar. 


Windows 


First, download the latest version of Python 2.7 from the official Website (httos://www.python.org/downloads/). 
Version is provided as an MSI package. To install it manually, just double-click the file. 


By default, Python installs to a directory: 


C:\Python27\ 


Warning: installation does not automatically modify the PATH environment variable. 


Assuming that your Python installation is in C:\Python27, add this to your PATH: 
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C:\Python27\ ;C:\Python27\Scripts\ 
Now to check if Python installation is valid write in cmd: 


python --version 


Python 2.x and 3.x Side-By-Side 
To install and use both Python 2.x and 3.x side-by-side on a Windows machine: 


1. Install Python 2.x using the MSI installer. 


o Ensure Python is installed for all users. 
© Optional: add Python to PATH to make Python 2.x callable from the command-line using python. 


2. Install Python 3.x using its respective installer. 


o Again, ensure Python is installed for all users. 

© Optional: add Python to PATH to make Python 3.x callable from the command-line using python. This 
may override Python 2.x PATH settings, so double-check your PATH and ensure it's configured to your 
preferences. 

o Make sure to install the py launcher for all users. 


Python 3 will install the Python launcher which can be used to launch Python 2.x and Python 3.x interchangeably 
from the command-line: 


P:\>py -3 

Python 3.6.1 (v3.6.1:69c@db5, Mar 21 2017, 17:54:52) [MSC v.198@ 32 bit (Intel)] on win32 
Type "help", "copyright", "credits" or "license" for more information. 

>>> 

C:\>py -2 

Python 2.7.13 (v2.7.13:a06454blafal, Dec 17 2016, 20:42:59) [MSC v.15@@ 32 Intel)] on win32 
Type "help", "copyright", "credits" or "license" for more information. 


>>> 


To use the corresponding version of pip for a specific Python version, use: 


C:\>py -3 -m pip -V 
pip 9.0.1 from C:\Python36\lib\site-packages (python 3.6) 


C:\>py -2 -m pip -V 
pip 9.0.1 from C:\Python27\lib\site-packages (python 2.7) 


Linux 
The latest versions of CentOS, Fedora, Red Hat Enterprise (RHEL) and Ubuntu come with Python 2.7. 
To install Python 2.7 on linux manually, just do the following in terminal: 


wget --no-check-certificate https://www.python.org/ftp/python/2.7.X/Python-2.7.X.tgz 
tar -xzf Python-2.7.X.tgz 

cd Python-2.7.X 

./configure 

make 
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sudo make install 


Also add the path of new python in PATH environment variable. If new python is in /root/python-2.7.X then run 
export PATH = SPATH:/root/python-2.7.X 


Now to check if Python installation is valid write in terminal: 
python --version 


Ubuntu (From Source) 


If you need Python 3.6 you can install it from source as shown below (Ubuntu 16.10 and 17.04 have 3.6 version in 
the universal repository). Below steps have to be followed for Ubuntu 16.04 and lower versions: 


sudo apt install build-essential checkinstall 

sudo apt install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm- 
dev libc6-dev libbz2-dev 

wget https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz 

tar xvf Python-3.6.1.tar.xz 

cd Python-3.6.1/ 

./configure --enable-optimizations 

sudo make altinstall 


macOS 


As we speak, macOS comes installed with Python 2.7.10, but this version is outdated and slightly modified from the 
regular Python. 


The version of Python that ships with OS X is great for learning but it’s not good for development. The 
version shipped with OS X may be out of date from the official current Python release, which is 
considered the stable production version. (Source) 


Install Homebrew: 


/usr/bin/ruby -e "S(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/instal1) " 
Install Python 2.7: 

brew install python 

For Python 3.x, use the command brew install python3 instead. 

Section 1.11: String function - strQ) and reprQ) 


There are two functions that can be used to obtain a readable representation of an object. 


repr(x) calls x.__repr__(): a representation of x. eval will usually convert the result of this function back to the 
original object. 
str(x) calls x.__str__(): a human-readable string that describes the object. This may elide some technical detail. 


repr() 
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For many types, this function makes an attempt to return a string that would yield an object with the same value 
when passed to eval(). Otherwise, the representation is a string enclosed in angle brackets that contains the name 
of the type of the object along with additional information. This often includes the name and address of the object. 


str() 


For strings, this returns the string itself. The difference between this and repr(object) is that str(object) does 
not always attempt to return a string that is acceptable to eval(). Rather, its goal is to return a printable or ‘human 
readable’ string. If no argument is given, this returns the empty string, ''. 


Example 1: 


S = w'o"w 
repr(s) # Output: '\'w\\\'o"w\'' 
str(s) # Output: ‘w\'o'w' 


eval(str(s)) == s # Gives a SyntaxError 
eval(repr(s)) == s # Output: True 
Example 2: 


import datetime 

today = datetime. datetime .now() 

str(today) # Output: '2016-@9-15 06:58:46.915060' 

repr(today) # Output: '‘datetime.datetime(2016, 9, 15, 6, 58, 46, 915000) ' 


When writing a class, you can override these methods to do whatever you want: 


class Represent(object): 


def ..init__(self, x, y): 
self.x, self.y = x, y 


def __repr__(self): 
return "Represent (x={}, y=\"{}\")".format(self.x, self.y) 


def __str__(self): 
return "Representing x as {} and y as {}".format(self.x, self.y) 


Using the above class we can see the results: 


r = Represent(1, "Hopper") 

print(r) # prints __str__ 

print(r.__repr__) # prints __repr__: ‘<bound method Represent.__repr__ of 
Represent(x=1, y="Hopper")>' 

rep = r.__repr__() # sets the execution of __repr__ to a new variable 
print(rep) # prints 'Represent(x=1, y="Hopper")' 

r2 = eval(rep) # evaluates rep 

print(r2) # prints __str__ from new object 

print(r2 == r) # prints 'False' because they are different objects 


Section 1.12: Installing external modules using pip 


pip is your friend when you need to install any package from the plethora of choices available at the python 
package index (PyPI). pip is already installed if you're using Python 2 >= 2.7.9 or Python 3 >= 3.4 downloaded from 
python.org. For computers running Linux or another *nix with a native package manager, pip must often be 
manually installed. 
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On instances with both Python 2 and Python 3 installed, pip often refers to Python 2 and pip3 to Python 3. Using 


pip will only install packages for Python 2 and pip3 will only install packages for Python 3. 
Finding / installing a package 
Searching for a package is as simple as typing 


S$ pip search <query> 
# Searches for packages whose name or summary contains <query> 


Installing a package is as simple as typing (in a terminal / commanda-prompt, not in the Python interpreter) 
$ pip install [package_name] # latest version of the package 

$ pip install [package_name]==x.x.x # specific version of the package 

$ pip install '[package_name]>=x.x.x' # minimum version of the package 

where x.x.x is the version number of the package you want to install. 


When your server is behind proxy, you can install package by using below command: 


$ pip --proxy http://<server address>:<port> install 
Upgrading installed packages 


When new versions of installed packages appear they are not automatically installed to your system. To get an 


overview of which of your installed packages have become outdated, run: 
$ pip list --outdated 

To upgrade a specific package use 

$ pip install [package_name] --upgrade 

Updating all outdated packages is not a standard functionality of pip. 


Upgrading pip 
You can upgrade your existing pip installation by using the following commands 


e On Linux or macOS x: 

S$ pip install -U pip 

You may need to use sudo with pip on some Linux Systems 
e On Windows: 

py -m pip install -U pip 


or 


python -m pip install -U pip 
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For more information regarding pip do read here. 


Section 1.13: Help Utility 


Python has several functions built into the interpreter. If you want to get information of keywords, built-in 
functions, modules or topics open a Python console and enter: 


>>> help() 


You will receive information by entering keywords directly: 


>>> help(help) 


or within the utility: 
help> help 


which will show an explanation: 


Help on Helper in module _sitebuiltins object: 


class Helper(builtins.object) 
Define the builtin 'help'. 


This is a wrapper around pydoc.help that provides a helpful message 
when 'help' is typed at the Python interactive prompt. 


| 

| 

| 

| 

| 

| Calling help() at the Python prompt starts an interactive help session. 
| Calling help(thing) prints help for the python object 'thing'. 
| 

| 

| 

| 

| 

| 

| 


Methods defined here: 
__call__(self, *args, **kwds) 
__repr_ (self) 


Data descriptors defined here: 


| 
| 
| _ dict __ 

| dictionary for instance variables (if defined) 
| 

| 

| 


__weakref__ 
list of weak references to the object (if defined) 


You can also request subclasses of modules: 
help(pymysql.connections) 

You can use help to access the docstrings of the different modules you have imported, e.g., try the following: 
>>> help(math) 


and you'll get an error 


>>> import math 
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>>> help(math) 


And now you will get a list of the available methods in the module, but only AFTER you have imported it. 


Close the helper with quit 


Goalkicker.com - Python® Notes for Professionals 


32 


Chapter 2: Python Data Types 


Data types are nothing but variables you use to reserve some space in memory. Python variables do not need an 
explicit declaration to reserve memory space. The declaration happens automatically when you assign a value to a 
variable. 


Section 2.1: String Data Type 


String are identified as a contiguous set of characters represented in the quotation marks. Python allows for either 
pairs of single or double quotes. Strings are immutable sequence data type, i.e each time one makes any changes 
to a string, completely new string object is created. 


a_str = ‘Hello World’ 

print(a_str) #output will be whole string. Hello World 
print(a_str[@]) #output will be first character. H 
print(a_str[@:5]) #output will be first five characters. Hello 


Section 2.2: Set Data Types 
Sets are unordered collections of unique objects, there are two types of set: 


1. Sets - They are mutable and new elements can be added once sets are defined 


basket = {'apple', ‘orange’, ‘apple', ‘pear', ‘orange’, ‘banana’ } 
print (basket) # duplicates will be removed 

> {'orange', ‘banana', 'pear', ‘apple'} 

a = set('abracadabra' ) 

print(a) # unique letters in a 

ater Mich Tey gn Ke 4 lh 

a.add('z') 

print(a) 


Se eae Lou Coes Bae Tray Bor} 


2. Frozen Sets - They are immutable and new elements cannot added after its defined. 


b = frozenset('asdfagsa' ) 


print(b) 

> frozenset({'f', ‘g', ‘d', ‘a', ‘s'}) 

cities = frozenset(["Frankfurt", "Basel", "Freiburg" ]) 
print(cities) 

> frozenset({'Frankfurt', 'Basel', ‘Freiburg’ }) 


Section 2.3: Numbers data type 


Numbers have four types in Python. Int, float, complex, and long. 


int_num = 180 #int value 
float_num = 10.2 #float value 
complex_num = 3.14j #complex value 


long_num = 1234567L #long value 
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Section 2.4: List Data Type 


A list contains items separated by commas and enclosed within square brackets [].lists are almost similar to arrays 
in C. One difference is that all the items belonging to a list can be of different data type. 


Tis tes (423) abcde da. 2. xd) #can be an array of any data type or single data type. 
list1 = ['hello', 'world' ] 

print(list) #will output whole list. [123, 'abcd',1@.2,'d'] 

print(list[@:2]) #will output first two element of list. [123, 'abcd'] 

print(list1 * 2) #will gave list1 two times. ['hello', ‘world’, ‘hello’, ‘world’ ] 
print(list + list1) #will gave concatenation of both the lists. 

[123, 'abcd',1@.2,'d', ‘hello’, ‘world’ ] 


Section 2.5: Dictionary Data Type 


Dictionary consists of key-value pairs. It is enclosed by curly braces {} and values can be assigned and accessed 
using square brackets[]. 


dic={'name':'red','age':10} 

print(dic) #will output all the key-value pairs. {'name':'red', 'age':10} 
print(dic['name']) #will output only value with ‘name' key. ‘red’ 
print(dic.values()) #will output list of values in dic. ['red’,1@] 
print(dic.keys()) #will output list of keys. ['name', ‘age’ ] 


Section 2.6: Tuple Data Type 


Lists are enclosed in brackets [] and their elements and size can be changed, while tuples are enclosed in 
parentheses () and cannot be updated. Tuples are immutable. 


tuple = (123, 'hello') 

tuple1 = (‘world’) 

print(tuple) #will output whole tuple. (123, 'hello') 
print(tuple[@]) #will output first value. (123) 
print(tuple + tuple1) #will output (123, 'hello', 'world') 
tuple[1]='update' #this will give you error. 
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Chapter 3: Indentation 


Section 3.1: Simple example 


For Python, Guido van Rossum based the grouping of statements on indentation. The reasons for this are explained 
in the first section of the "Design and History Python FAQ". Colons, :, are used to declare an indented code block, 
such as the following example: 


class ExampleClass: 
#Every function belonging to a class must be indented equally 
def __init__(self): 
name = "example" 


def someFunction(self, a): 
#Notice everything belonging to a function must be indented 
if <a> 5: 
return True 
else: 
return False 


#If a function is not indented to the same level it will not be considers as part of the parent class 
def separateFunction(b): 
for i in b: 
#Loops are also indented and nested conditions start a new indentation 
if i == 
return True 
return False 


separateFunction([2,3,5,6,1]) 


Spaces or Tabs? 


The recommended indentation is 4 spaces but tabs or spaces can be used so long as they are consistent. Do not 
mix tabs and spaces in Python as this will cause an error in Python 3 and can causes errors in Python 2. 


Section 3.2: How Indentation is Parsed 


Whitespace is handled by the lexical analyzer before being parsed. 


The lexical analyzer uses a stack to store indentation levels. At the beginning, the stack contains just the value 0, 
which is the leftmost position. Whenever a nested block begins, the new indentation level is pushed on the stack, 
and an "INDENT" token is inserted into the token stream which is passed to the parser. There can never be more 
than one "INDENT" token in a row (IndentationError). 


When a line is encountered with a smaller indentation level, values are popped from the stack until a value is on top 
which is equal to the new indentation level (if none is found, a syntax error occurs). For each value popped, a 
"DEDENT" token is generated. Obviously, there can be multiple "DEDENT" tokens in a row. 


The lexical analyzer skips empty lines (those containing only whitespace and possibly comments), and will never 
generate either "INDENT" or "DEDENT" tokens for them. 


At the end of the source code, "DEDENT" tokens are generated for each indentation level left on the stack, until just 
the 0 is left. 


For example: 
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if foo: 


if bar: 
x = 42 
else: 
print foo 


is analyzed as: 


<if> <foo> <:> [0] 
<INDENT> <if> <bar> <:> [8, 4] 
<INDENT> <x> <=> <42> [e, 4, 8] 
<DEDENT> <DEDENT> <else> <:> [8] 
<INDENT> <print> <foo> [@, 2] 
<DEDENT> 


The parser than handles the "INDENT" and "DEDENT" tokens as block delimiters. 


Section 3.3: Indentation Errors 


The spacing should be even and uniform throughout. Improper indentation can cause an IndentationError or 
cause the program to do something unexpected. The following example raises an IndentationError: 


a=/7 
Lf a oe 
print "foo" 
else: 
print "bar" 
print "done" 


Or if the line following a colon is not indented, an IndentationError will also be raised: 


if True: 
print "true" 


If you add indentation where it doesn't belong, an IndentationError will be raised: 


if TGwue: 
a = 6 
be=.5 


If you forget to un-indent functionality could be lost. In this example None is returned instead of the expected False: 


def isEven(a): 
if a%2 ==0: 
return True 
#this next line should be even with the if 
return False 
print isEven(7) 
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Chapter 4: Comments and Documentation 


Section 4.1: Single line, inline and multiline comments 


Comments are used to explain code when the basic code itself isn't clear. 
Python ignores comments, and so will not execute code in there, or raise syntax errors for plain English sentences. 
Single-line comments begin with the hash character (#) and are terminated by the end of line. 
e Single line comment: 
# This is a single line comment in Python 
e Inline comment: 
print("Hello World") # This line prints "Hello World" 


¢ Comments spanning multiple lines have """ or ''' on either end. This is the same as a multiline string, but 
they can be used as comments: 


This type of comment spans multiple lines. 
These are mostly used for documentation of functions, classes and modules. 


Section 4.2: Programmatically accessing docstrings 


Docstrings are - unlike regular comments - stored as an attribute of the function they document, meaning that you 
can access them programmatically. 


An example function 


def func(): 
"""This is a function that does nothing at all""" 


return 
The docstring can be accessed using the __doc__ attribute: 


print(func.__doc__) 
This is a function that does nothing at all 
help( func) 


Help on function func in module __main__: 
func() 


This is a function that does nothing at all 
Another example function 
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function.__doc__ is just the actual docstring as a string, while the help function provides general information 
about a function, including the docstring. Here's a more helpful example: 


def greet(name, greeting="Hello"): 
"""Print a greeting to the user “name~ 


Optional parameter “greeting can change what they're greeted with.""" 


print("{} {}".format(greeting, name)) 


help(greet) 


Help on function greet in module __main__: 
greet(name, greeting='Hello' ) 


Print a greeting to the user name 
Optional parameter greeting can change what they're greeted with. 


Advantages of docstrings over regular comments 
Just putting no docstring or a regular comment in a function makes it a lot less helpful. 
def greet(name, greeting="Hello"): 
# Print a greeting to the user ‘name~ 


# Optional parameter ‘greeting’ can change what they're greeted with. 


print("{} {}".format(greeting, name) ) 


print(greet.__doc__) 
None 


help(greet) 


Help on function greet in module main: 


greet(name, greeting='Hello' ) 


Section 4.3: Write documentation using docstrings 


A docstring is a multi-line comment used to document modules, classes, functions and methods. It has to be the 
first statement of the component it describes. 


def hello(name) : 
"""Greet someone. 


Print a greeting ("Hello") for the person with the given name. 


print("Hello "+name) 


class Greeter: 
"""An object used to greet people. 
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It contains multiple greeting functions for several languages 
and times of the day. 


The value of the docstring can be accessed within the program and is - for example - used by the help command. 


Syntax conventions 
PEP 257 


PEP 257 defines a syntax standard for docstring comments. It basically allows two types: 
e One-line Docstrings: 
According to PEP 257, they should be used with short and simple functions. Everything is placed in one line, e.g: 


def hello(): 
"""Say hello to your friends. 
print("Hello my friends!") 


The docstring shall end with a period, the verb should be in the imperative form. 
e Multi-line Docstrings: 
Multi-line docstring should be used for longer, more complex functions, modules or classes. 


def hello(name, language="en"): 
"""Say hello to a person. 


Arguments: 
name: the name of the person 
language: the language in which the person should be greeted 


print(greeting[language]+" “+name) 


They start with a short summary (equivalent to the content of a one-line docstring) which can be on the same line 


as the quotation marks or on the next line, give additional detail and list parameters and return values. 


Note PEP 257 defines what information should be given within a docstring, it doesn't define in which format it 
should be given. This was the reason for other parties and documentation parsing tools to specify their own 
standards for documentation, some of which are listed below and in this question. 


Sphinx 


Sphinx is a tool to generate HTML based documentation for Python projects based on docstrings. Its markup 


language used is reStructuredText. They define their own standards for documentation, pythonhosted.org hosts a 


very good description of them. The Sphinx format is for example used by the pyCharm IDE. 


A function would be documented like this using the Sphinx/reStructuredText format: 


def hello(name, language="en"): 
"""Say hello to a person. 


:param name: the name of the person 

:type name: str 

:param language: the language in which the person should be greeted 
:type language: str 
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:return: a number 
irtype: int 


print(greeting[language]+" “+name) 


return 4 


Google Python Style Guide 


Google has published Google Python Style Guide which defines coding conventions for Python, including 
documentation comments. In comparison to the Sphinx/reST many people say that documentation according to 
Google's guidelines is better human-readable. 


The pythonhosted.org page mentioned above also provides some examples for good documentation according to 
the Google Style Guide. 


Using the Napoleon plugin, Sphinx can also parse documentation in the Google Style Guide-compliant format. 


A function would be documented like this using the Google Style Guide format: 


def hello(name, language="en"): 
"""Say hello to a person. 


Args: 
name: the name of the person as string 
language: the language code string 


Returns: 
A number. 


print(greeting[language]+" “+name) 


return 4 


Goalkicker.com - Python® Notes for Professionals 40 


VIDEO: Complete Python 
Bootcamp: Go trom zero 
to hero in Python 3 


Learn Python like a Professional! Start from the 
basics and go all the way to creating your own 
applications and games! 


COMPLETE PYTHON S 
BOOTCAMP 


v Learn to use Python professionally, learning both Python 2 and Python 3! 

Y¥ Create games with Python, like Tic Tac Toe and Blackjack! 

¥ Learn advanced Python features, like the collections module and how to work with timestamps! 
v Learn to use Object Oriented Programming with classes! 

v Understand complex topics, like decorators. 

v Understand how to use both the Jupyter Notebook and create .py files 

Y Get an understanding of how to create GUls in the Jupyter Notebook system! 

¥Y Build a complete understanding of Python from the ground up! 


Watch Today — 


Chapter 5: Date and Time 


ater 5.1: Parsing a string into a timezone aware datetime 
object 


Python 3.2+ has support for %z format when parsing a string into a datetime object. 
UTC offset in the form +HHMM or -HHMM (empty string if the object is naive). 


Python 3.x Version = 3.2 


import datetime 
dt = datetime.datetime.strptime("2016-04-15T08:27:18-9500", "%Y-%m-%dT%H :%M :%S%z" ) 


For other versions of Python, you can use an external library such as dateutil, which makes parsing a string with 
timezone into a datetime object is quick. 


import dateutil.parser 
dt = dateutil.parser.parse("2016-04-15T@8 :27:18-0580" ) 


The dt variable is now a datetime object with the following value: 


datetime.datetime(2016, 4, 15, 8, 27, 18, tzinfo=tzoffset(None, -1800@)) 


Section 5.2: Constructing timezone-aware datetimes 


By default all datetime objects are naive. To make them timezone-aware, you must attach a tzinfo object, which 
provides the UTC offset and timezone abbreviation as a function of date and time. 


Fixed Offset Time Zones 


For time zones that are a fixed offset from UTC, in Python 3.2+, the datetime module provides the timezone Class, a 
concrete implementation of tzinfo, which takes a timedelta and an (optional) name parameter: 


Python 3.x Version = 3.2 


from datetime import datetime, timedelta, timezone 
JST = timezone(timedelta(hours=+9) ) 


dt = datetime(2015, 1, 1, 12, @, 8, tzinfo=JST) 
print(dt) 
# 2015-01-91 12:00:60+069:00 


print(dt.tzname() ) 
# UTC+O9 00 


dt = datetime(2015, 1, 1, 12, ®, 8, tzinfo=timezone(timedelta(hours=9), 'JST')) 
print(dt.tzname) 
He OSs 


For Python versions before 3.2, it is necessary to use a third party library, such as dateutil. dateutil provides an 
equivalent class, tzoffset, which (as of version 2.5.3) takes arguments of the form dateutil.tz.tzoffset(tzname, 
offset), where offset is specified in seconds: 


Python 3.x Version < 3.2 
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Python 2.x Version < 2.7 


from datetime import datetime, timedelta 
from dateutil import tz 


JST = tz.tzoffset('JST', 9 * 3600) # 368@ seconds per hour 
dt = datetime(2015, 1, 1, 12, ®, tzinfo=JST) 

print(dt) 

# 2015-01-01 12:00:00+09:00 

print (dt.tzname) 

# 'JST' 


Zones with daylight savings time 


For zones with daylight savings time, python standard libraries do not provide a standard class, so it is necessary to 
use a third party library. pytz and dateutil are popular libraries providing time zone classes. 


In addition to static time zones, dateutil provides time zone classes that use daylight savings time (see the 
documentation for the tz module). You can use the tz. gettz() method to get a time zone object, which can then 
be passed directly to the datetime constructor: 


from datetime import datetime 

from dateutil import tz 

local = tz.gettz() # Local time 

PT = tz.gettz('US/Pacific') # Pacific time 


dt_l = datetime(2015, 1, 1, 12, tzinfo=local) # I am in EST 

dt_pst = datetime(2015, 1, 1, 12, tzinfo=PT) 

dt_pdt = datetime(2015, 7, 1, 12, tzinfo=PT) # DST is handled automatically 
print(dt_1) 

# 2015-01-01 12:00:00-05:00 

print(dt_pst) 

# 2015-01-01 12:00:00-08:00 

print (dt_pdt) 

# 2015-07-01 12:00:00-07:00 


CAUTION: As of version 2.5.3, dateutil does not handle ambiguous datetimes correctly, and will always default to 
the /ater date. There is no way to construct an object with a dateutil timezone representing, for example 
2015-11-81 1:38 EDT-4, since this is during a daylight savings time transition. 


All edge cases are handled properly when using pytz, but pytz time zones should not be directly attached to time 
zones through the constructor. Instead, a pytz time zone should be attached using the time zone's localize 
method: 


from datetime import datetime, timedelta 
import pytz 


PT = pytz.timezone('US/Pacific' ) 

dt_pst = PT.localize(datetime(20@15, 1, 1, 12)) 
dt_pdt = PT.localize(datetime(2015, 11, 1, @, 3@)) 
print(dt_pst) 

# 2015-01-01 12:00:00-08:00 

print (dt_pdt) 

# 2015-11-01 00:30:00-07:00 


Be aware that if you perform datetime arithmetic on a pytz-aware time zone, you must either perform the 
calculations in UTC (if you want absolute elapsed time), or you must call normalize() on the result: 
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dt_new = dt_pdt + timedelta(hours=3) # This should be 2:3@ AM PST 
print(dt_new) 

# 2015-11-01 03:30:00-07:00 

dt_corrected = PT.normalize(dt_new) 

print(dt_corrected) 

# 2015-11-01 02:30:00-08:00 


Section 5.3: Computing time differences 
the timedelta module comes in handy to compute differences between times: 


from datetime import datetime, timedelta 
now = datetime.now() 
then = datetime(2016, 5, 23) # datetime.datetime(2@16, 95, 23, 9, @, @) 


Specifying time is optional when creating a new datetime object 
delta = now-then 
delta is of type timedelta 


print(delta.days) 

# 60 
print(delta.seconds) 
# 40826 


To get n day's after and n day's before date we could use: 
n day's after date: 


def get_n_days_after_date(date_format="%d %B %Y", add_days=120@): 


date_n_days_after = datetime.datetime.now() + timedelta(days=add_days) 
return date_n_days_after.strftime(date_format) 


n day's before date: 


def get_n_days_before_date(self, date_format="%d %B %Y", days_before=12@): 


date_n_days_ago = datetime.datetime.now() - timedelta(days=days_before) 
return date_n_days_ago.strftime(date_format ) 


Section 5.4: Basic datetime objects usage 


The datetime module contains three primary types of objects - date, time, and datetime. 


import datetime 


# Date object 
today = datetime.date.today() 
new_year = datetime.date(2017, 01, 01) #datetime.date(2017, 1, 1) 


# Time object 
noon = datetime.time(12, 8, 0) #datetime.time(12, 4) 


# Current datetime 
now = datetime.datetime.now() 
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# Datetime object 
millenium_turn = datetime.datetime(2000, 1, 1, 0, 9, @) #datetime.datetime(2@00, 1, 1, @, @) 


Arithmetic operations for these objects are only supported within same datatype and performing simple arithmetic 
with instances of different types will result in a TypeError. 


# subtraction of noon from today 
noon- today 
Traceback (most recent call last): 
File "<stdin>", line 1, in <module> 
TypeError: unsupported operand type(s) for -: 'datetime.time' and 'datetime.date' 
However, it is straightforward to convert between types. 


# Do this instead 
print('Time since the millenium at midnight: ', 

datetime.datetime(today.year, today.month, today.day) - millenium_turn) 
# Or this 


print('Time since the millenium at noon: ', 
datetime.datetime.combine(today, noon) - millenium_turn) 


Section 5.5: Switching between time zones 
To switch between time zones, you need datetime objects that are timezone-aware. 


from datetime import datetime 
from dateutil import tz 


utc = tz.tzutc() 
local = tz.tzlocal() 


utc_now = datetime.utcnow() 
utc_now # Not timezone-aware. 


utc_now = utc_now.replace(tzinfo=utc) 
utc_now # Timezone-aware. 


local_now = utc_now.astimezone(local) 
local_now # Converted to local time. 


Section 5.6: Simple date arithmetic 


Dates don't exist in isolation. It is common that you will need to find the amount of time between dates or 
determine what the date will be tomorrow. This can be accomplished using timedelta objects 


import datetime 


today = datetime.date.today() 
print('Today:', today) 


yesterday = today - datetime.timedelta(days=1 ) 
print('Yesterday:', yesterday) 


tomorrow = today + datetime.timedelta(days=1) 
print('Tomorrow:', tomorrow) 


print('Time between tomorrow and yesterday:', tomorrow - yesterday) 
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This will produce results similar to: 


Today: 2016-04-15 

Yesterday: 2016-04-14 

Tomorrow: 2016-04-16 

Difference between tomorrow and yesterday: 2 days, 0:00:00 


Section 5.7: Converting timestamp to datetime 
The datetime module can convert a POSIX timestamp to a ITC datetime object. 


The Epoch is January 1st, 1970 midnight. 


import time 
from datetime import datetime 
seconds_since_epoch=time.time() #1469182681.709 


utc_date=datetime.utcfromtimestamp(seconds_since_epoch) #datetime.datetime(2016, 7, 22, 10, 18, 
709000) 


Section 5.8: Subtracting months from a date accurately 
Using the calendar module 


import calendar 
from datetime import date 


def monthdelta(date, delta): 
m, y = (date.month+delta) % 12, date.year + ((date.month)+delta-1) // 12 
if not m: m = 12 
d = min(date.day, calendar.monthrange(y, m)[1]) 
return date.replace(day=d,month=m, year=y) 


next_month = monthdelta(date.today(), 1) #datetime.date(2016, 10, 23) 
Using the dateutils module 


import datetime 
import dateutil.relativedelta 


d = datetime. datetime.strptime("2013-03-31", "%Y-%m-%d" ) 
d2 = d - dateutil.relativedelta.relativedelta(months=1) #datetime.datetime(20@13, 2, 28, @, @) 


Section 5.9: Parsing an arbitrary ISO 8601 timestamp with 
minimal libraries 


Python has only limited support for parsing ISO 8601 timestamps. For strptime you need to know exactly what 
format it is in. As a complication the stringification of a datetime is an ISO 8601 timestamp, with space as a 
separator and 6 digit fraction: 


str(datetime.datetime(2016, 7, 22, 9, 25, 59, 555555)) 
# '2016-@7-22 09:25:59.555555' 


but if the fraction is 0, no fractional part is output 
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str(datetime.datetime(2016, 7, 22, 9, 25, 59, @)) 
# '2016-@7-22 09:25:59' 


But these 2 forms need a different format for strptime. Furthermore, strptime' does not support at all 
parsing minute timezones that have atin it, thus2016-07-22 09:25:59+0300can be parsed, but the 
standard format2016-07-22 09:25:59+03:00° cannot. 


There is a single-file library called iso8601 which properly parses ISO 8601 timestamps and only them. 
It supports fractions and timezones, and the T separator all with a single function: 


import iso08601 

iso8601 .parse_date('2016-@7-22 89:25:59') 

# datetime.datetime(2016, 7, 22, 9, 25, 59, tzinfo=<iso860@1.Utc>) 

iso8601 .parse_date('2016-@7-22 09:25:59+03:00') 

# datetime.datetime(2016, 7, 22, 9, 25, 59, tzinfo=<FixedOffset '+0@3:0@' ...>) 
iso8601.parse_date('2016-@7-22 09:25:59Z') 

# datetime.datetime(2016, 7, 22, 9, 25, 59, tzinfo=<iso860@1.Utc>) 

iso8601 .parse_date('2016-07-22TO9 :25:59 .@00111+03 :00' ) 

# datetime.datetime(2016, 7, 22, 9, 25, 59, 111, tzinfo=<FixedOffset '+@3:0@' ...>) 


If no timezone is set, iso8601.parse_date defaults to UTC. The default zone can be changed with default_zone 
keyword argument. Notably, if this is None instead of the default, then those timestamps that do not have an 
explicit timezone are returned as naive datetimes instead: 


iso8601 .parse_date('2016-@7-22T09:25:59', default_timezone=None) 
# datetime.datetime(2016, 7, 22, 9, 25, 59) 

iso8601 .parse_date('2016-07-22T09:25:59Z' , default_timezone=None) 
# datetime.datetime(2016, 7, 22, 9, 25, 59, tzinfo=<iso8601.Utc>) 


Section 5.10: Get an ISO 8601 timestamp 


Without timezone, with microseconds 
from datetime import datetime 
datetime.now().isoformat() 

# Out: '2016-@7-31T23 :08:20.886783' 


With timezone, with microseconds 


from datetime import datetime 
from dateutil.tz import tzlocal 


datetime.now(tzlocal()).isoformat() 
# Out: '2016-@7-317T23 :09:43.535074-07:00' 


With timezone, without microseconds 


from datetime import datetime 
from dateutil.tz import tzlocal 


datetime.now(tzlocal()).replace(microsecond=@) .isoformat() 
# Out: '2016-@7-31T23:10:30-07:00' 


See ISO 8601 for more information about the ISO 8601 format. 


Section 5.11: Parsing a string with a short time zone name into 
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a timezone aware datetime object 


Using the dateutil library as in the previous example on parsing timezone-aware timestamps, it is also possible to 
parse timestamps with a specified "short" time zone name. 


For dates formatted with short time zone names or abbreviations, which are generally ambiguous (e.g. CST, which 
could be Central Standard Time, China Standard Time, Cuba Standard Time, etc - more can be found here) or not 

necessarily available in a standard database, it is necessary to specify a mapping between time zone abbreviation 
and tzinfo object. 


from dateutil import tz 
from dateutil.parser import parse 


ET = tz.gettz('US/Eastern' ) 
CT = tz.gettz('US/Central' ) 
MT = tz.gettz('US/Mountain’ ) 
PT = tz.gettz('US/Pacific' ) 


USmtZInhos =) (wCSik = Cl JGDie = 1Gh, 
CESiiecs Ele EDI: Ee 
‘MST’: MT, ‘MDT': MT, 
sPSie Pl sPDiie Pi} 


dt_est = parse('2014-01-02 04:00:00 EST', tzinfos=us_tzinfos) 
dt_pst = parse('2016-@3-11 16:00:00 PST', tzinfos=us_tzinfos) 


After running this: 


dt_est 

# datetime.datetime(2014, 1, 2, 4, @, tzinfo=tzfile('/usr/share/zoneinfo/US/Eastern’' ) ) 
dt_pst 

# datetime.datetime(2016, 3, 11, 16, @, tzinfo=tzfile('/usr/share/zoneinfo/US/Pacific' ) ) 


It is worth noting that if using a pytz time zone with this method, it will not be properly localized: 


from dateutil.parser import parse 
import pytz 


EST = pytz.timezone('America/New_York' ) 
dt = parse('2014-02-03 09:17:00 EST', tzinfos={'EST': EST}) 


This simply attaches the pytz time zone to the datetime: 


dt.tzinfo # Will be in Local Mean Time! 
# <DstTzInfo ‘America/New_York' LMT-1 day, 19:04:0@ STD> 


If using this method, you should probably re-localize the naive portion of the datetime after parsing: 


dt_fixed = dt.tzinfo.localize(dt.replace(tzinfo=None) ) 
dt_fixed.tzinfo # Now it's EST. 
# <DstTzInfo '‘America/New_York' EST-1 day, 19:@0:@@ STD>) 


Section 5.12: Fuzzy datetime parsing (extracting datetime out 
of a text) 


It is possible to extract a date out of a text using the dateutil parser in a "fuzzy" mode, where components of the 
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string not recognized as being part of a date are ignored. 


from dateutil.parser import parse 


dt = parse("Today is January 1, 2047 at 8:21:@@0AM", fuzzy=True) 
print(dt) 


dt is now a datetime object and you would see datetime.datetime(2047, 1, 1, 8, 21) printed. 


Section 5.13: Iterate over dates 


Sometimes you want to iterate over a range of dates from a start date to some end date. You can do it using 
datetime library and timedelta object: 


import datetime 


# The size of each step in days 
day_delta = datetime.timedelta(days=1) 


start_date = datetime.date.today() 
end_date = start_date + 7*day_delta 


for i in range((end_date - start_date).days): 
print(start_date + ixday_delta) 


Which produces: 


2016-07-21 
2016-07-22 
2016-07-23 
2016-07-24 
2016-07-25 
2016-07-26 
2016-07-27 
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Chapter 6: Date Formatting 


Section 6.1: Time between two date-times 


from datetime import datetime 


a = datetime(2016,10,06,0,0,0) 
b = datetime(2016,10,01,23,59,59) 
a-b 


# datetime.timedelta(4, 1) 


(a-b) .days 

#4 

(a-b) .total_seconds() 
# 518399.0 


Section 6.2: Outputting datetime object to string 
Uses C standard format codes. 


from datetime import datetime 

datetime_for_string = datetime(2016,10,1,0,@) 
datetime_string_format = ‘'%b %d %Y, %H:%M:%S' 
datetime.strftime(datetime_for_string, datetime_string_format) 
# Oct 61 2016, 80:00:00 


Section 6.3: Parsing string to datetime object 
Uses C standard format codes. 


from datetime import datetime 

datetime_string = ‘Oct 1 2016, 00:00:00' 
datetime_string_format = ‘'%b %d %Y, %H:%M:%S' 
datetime.strptime(datetime_string, datetime_string_format) 
# datetime.datetime(2016, 16, 1, @, @) 
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Chapter 7: Enum 
Section 7.1: Creating an enum (Python 2.4 through 3.3) 


Enums have been backported from Python 3.4 to Python 2.4 through Python 3.3. You can get this the enum34 
backport from PyPI. 


pip install enum34 
Creation of an enum is identical to how it works in Python 3.4+ 


from enum import Enum 


class Color(Enum) : 
red = 1 
green = 2 
blue = 3 


print(Color.red) # Color.red 


print(Color(1)) # Color.red 
print(Color['red']) # Color.red 


Section 7.2: Iteration 
Enums are iterable: 


class Color(Enum): 
red = 1 
green = 2 
blue = 3 


[c for c in Color] # [<Color.red: 1>, <Color.green: 2>, <Color.blue: 3>] 
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Chapter 8: Set 


Section 8.1: Operations on sets 


with other sets 


# Intersection 
{1, 2, 3, 4, 5}.intersection({3, 4, 5, 6}) # {3, 4, 5} 


{1, 2, 3, 4; 5} & {3, 4, 5; 6} # {3, 4, 5} 
# Union 

{ile 22d a Sy UNTO oA oO) ome Nie Zoe 4 eo Os 
(U2, So Ae oye AS, 4 0, Gy el Cp wn Ze By TO: 


# Difference 
11, 2, 3) 4). dififference({2, 37 S))) 4 {1) 4} 
aly De a 4} = {2 2 5} # dite, 4} 


# Symmetric difference with 
{1, 2, 3, 4}.symmetric_difference({2, 3, 5}) # {1, 4, 5} 
(i 25 oie A eS 2s eo. # {1, 5} 


A 


# Superset check 
{1, 2}.issuperset({1, 2, 3}) # False 
ily op BS AU Pon SEY # False 


# Subset check 
{1, 2}.issubset({1, 2, 3}) # True 
{1, 2} <= {1 2, 3} # True 


# Disjoint check 


{1, 2}.isdisjoint({3, 4}) # True 
{1, 2}.isdisjoint({1, 4}) # False 


with single elements 


# Existence check 

2 in {12,3} # True 

Aaa ly 2: cai), # False 

4 not in {1,2,3} # True 

# Add and Remove 

Sear dlzissy 

s.add(4) #s == {1,2,3,4} 
s.discard(3) HaSu—— ere 
s.discard(5) HSu—— elena 
s.remove(2) #s == {1,4} 
s.remove(2) # KeyError! 


Set operations return new sets, but have the corresponding in-place versions: 


method in-place operation in-place method 
union s|=t update 
intersection s&=t intersection_update 
difference s-=t difference_update 
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symmetric_difference s 4=t symmetric_difference_update 


For example: 


Coe Gil 72}; 
s.update({3, 4}) Tay ine Pan ele eit 


Section 8.2: Get the unique elements of a list 


Let's say you've got a list of restaurants -- maybe you read it from a file. You care about the unique restaurants in 
the list. The best way to get the unique elements from a list is to turn it into a set: 


restaurants = ["McDonald's", "Burger King", "McDonald's", "Chicken Chicken" ] 
unique_restaurants = set(restaurants) 


print(unique_restaurants) 
# prints {'Chicken Chicken', "McDonald's", ‘Burger King’ } 


Note that the set is not in the same order as the original list; that is because sets are unordered, just like dicts. 


This can easily be transformed back into a List with Python's built in list function, giving another list that is the 
same list as the original but without duplicates: 


list(unique_restaurants) 
# ['Chicken Chicken', "McDonald's", ‘Burger King’ ] 


It's also common to see this as one line: 


# Removes all duplicates and returns another list 
list(set(restaurants) ) 


Now any operations that could be performed on the original list can be done again. 


Section 8.3: Set of Sets 
{{1,2}, {3,4}} 


leads to: 


TypeError: unhashable type: ‘set’ 


Instead, use frozenset: 


{frozenset({1, 2}), frozenset({3, 4})} 


Section 8.4: Set Operations using Methods and Builtins 


We define two sets aandb 


eat ate a ‘il; 25 2; 3; 4} 
eeeibi = (3; 3, 4, 4, 5) 


NOTE: {1} creates a set of one element, but {} creates an empty dict. The correct way to create an 


empty set is set(). 
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Intersection 
a.intersection(b) returns a new set with elements present in both a and b 


>>> a.intersection(b) 
{3, 4} 


Union 
a.union(b) returns a new set with elements present in either a and b 


>>> a.union(b) 
{1, 2 3; 4, Sy 


Difference 
a.difference(b) returns a new set with elements present in a but not in b 
>>> a.difference(b) 


tiles 


>>> b.difference(a) 
{5} 
Symmetric Difference 


a.symmetric_difference(b) returns a new set with elements present in either a or b but not in both 


>>> a.symmetric_difference(b) 


{ie 2553} 
>>> b.symmetric_difference(a) 
belts 5 ol 


NOTE: a.symmetric_difference(b) == b.symmetric_difference(a) 
Subset and superset 

c.issubset(a) tests whether each element of c is in a. 
a.issuperset(c) tests whether each element of c is ina. 

age ee 

>>> c.issubset(a) 

True 


>>> a.issuperset(c) 
True 


The latter operations have equivalent operators as shown below: 


Method Operator 
a.intersection(b) a&b 
a.union(b) a|b 
a.difference(b) a-b 
a.symmetric_difference(b) a * b 
a.issubset(b) a <= )b 
a.issuperset(b) a >= )b 


Disjoint sets 
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Sets a and d are disjoint if no element in a is also in d and vice versa. 


>>> d = {5, 6} 

>>> a.isdisjoint(b) # {2, 3, 4} are in both sets 
False 

>>> a.isdisjoint(d) 

True 


# This is an equivalent check, but less efficient 
>>> len(a & d) == @ 
True 


# This is even less efficient 
>>> a &d == set() 
True 


Testing membership 
The builtin in keyword searches for occurances 
>>> 1 ina 

True 


>>> 6 ina 
False 


Length 
The builtin len() function returns the number of elements in the set 


>>> len(a) 
4 
>>> len(b) 
=) 


Section 8.5: Sets versus multisets 


Sets are unordered collections of distinct elements. But sometimes we want to work with unordered collections of 
elements that are not necessarily distinct and keep track of the elements’ multiplicities. 


Consider this example: 
>>> SeitAe= (icc ae Dien Cuay 
>>> setA 


ser( [ta 3p Gi, well) 


By saving the strings 'a', 'b', 'b', ‘c' into aset data structure we've lost the information on the fact that ‘b' 
occurs twice. Of course saving the elements to a list would retain this information 


SS listA = Pe le el 

>>> listA 

era Aly Tay unl 

but a list data structure introduces an extra unneeded ordering that will slow down our computations. 


For implementing multisets Python provides the Counter class from the collections module (starting from version 
2.7): 


Python 2.x Version = 2.7 
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>>> from collections import Counter 

>>> counterA = Counter(['a','b','b','c']) 
>>> counterA 

Counter ({ be: 2) sare 1, <c = 1}) 


Counter is a dictionary where where elements are stored as dictionary keys and their counts are stored as 
dictionary values. And as all dictionaries, it is an unordered collection. 
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Chapter 9: Simple Mathematical Operators 


Numerical types and their metaclasses 
The numbers module contains the abstract metaclasses for the numerical types: 


subclasses numbers.Number numbers.Integral numbers.Rational numbers.Real numbers.Complex 


bool v v v v v 
int v v v v v 
fractions.Fraction ¥ - v v v 
float v - - v v 
complex v - - - v 


decimal.Decimal ¥ - = - = 


Python does common mathematical operators on its own, including integer and float division, multiplication, 
exponentiation, addition, and subtraction. The math module (included in all standard Python versions) offers 
expanded functionality like trigonometric functions, root operations, logarithms, and many more. 


Section 9.1: Division 


Python does integer division when both operands are integers. The behavior of Python's division operators have 
changed from Python 2.x and 3.x (see also Integer Division ). 


aDieCh dee oye, 2 Oe oa 


Python 2.x Version < 2.7 


In Python 2 the result of the'/' operator depends on the type of the numerator and denominator. 


a/b #=7 
a/c #=1.5 
d/b # = -2 
b/a #= 6 
d/e # = -7 


Note that because both a and b are ints, the result is an int. 
The result is always rounded down (floored). 
Because c is a float, the result ofa / cisa float. 
You can also use the operator module: 
import operator the operator module provides 2-argument arithmetic functions 


# 
operator.div(a, b) a | 
operator.__div__(a, b) # = 1 


Python 2.x Version = 2.2 
What if you want float division: 


Recommended: 
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from 
a/b 
a //b 


# 
# 


1 


_future__ import division # applies Python 3 style division to the entire module 
Ve 


5 


Okay (if you don't want to apply to the whole module): 


# 


pra 


# 


from operator import truediv 
# — 


truediv(a, b) 


tle 


the 
ile 
1 


5 
2) 
2) (careful with order of operations) 


5 


Not recommended (may raise TypeError, eg if argument is complex): 


float(a) / b 
a / float(b) 


Python 2.x Version = 2.2 


# 
# 


a 
an 


The '//' operator in Python 2 forces floored division regardless of type. 


a //b # 
a//c # 


Python 3.x Version = 3.0 


In Python 3 the / operator performs 'true' division regardless of types. The // operator performs floor division and 


maintains type. 


o 9 0 ® 
SS 
om on 


import operator 

operator.truediv(a, b) 
operator.floordiv(a, b) 
operator.floordiv(a, c) 


we RR 


we RR 


3A oO = 


the operator module provides 2-argument arithmetic functions 
= 1.5 


Possible combinations (builtin types): 


e int and int (gives an int in Python 2 and a float in Python 3) 


int and float (gives a float) 


e int and complex (gives a complex) 
e float and float (gives a float) 


float and complex (gives a complex) 
complex and complex (gives a complex) 


See PEP 238 for more information. 


Section 9.2: Addition 


# Using the "+" operator: 
# = 3 


a+b 
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# Using the "in-place" "+=" operator to add and assign: 


a += b # a = 3 (equivalent to a = a+b) 
import operator # contains 2 argument arithmetic functions for the examples 
operator.add(a, b) #= 5 since a is set to 3 right before this line 


# The "+=" operator is equivalent to: 
a = operator.iadd(a, b) # a= 5 since a is set to 3 right before this line 


Possible combinations (builtin types): 


e int and int (gives an int) 

int and float (gives a float) 

e int and complex (gives a complex) 
float and float (gives a float) 

float and complex (gives a complex) 

* complex and complex (gives a complex) 


Note: the + operator is also used for concatenating strings, lists and tuples: 


"first string " + "second string" # = ‘first string second string’ 


[teen Sl + (4s 5.61 Hos [i293 4,05 OF 


Section 9.3: Exponentiation 


aba 2S 
(a ** b) a= tO 
pow(a, b) # = 8 


import math 


math.pow(a, b) # = 8.0 (always float; does not allow complex results) 
import operator 
operator.pow(a, b) # = 8 


Another difference between the built-in pow and math. pow is that the built-in pow can accept three arguments: 


pow(2, 3, 2) # @, calculates (2 ** 3) % 2, but as per Python docs, 
# does so more efficiently 


Special functions 
The function math. sqrt(x) calculates the square root of x. 


import math 
import cmath 


c=4 
math.sqrt(c) # = 2.@ (always float; does not allow complex results) 
cmath.sqrt(c) # = (2+0j) (always complex) 


To compute other roots, such as a cube root, raise the number to the reciprocal of the degree of the root. This 
could be done with any of the exponential functions or operator. 
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import math 

x = 8 

math.pow(x, 1/3) # evaluates to 2.0 
x**(1/3) # evaluates to 2.0 


The function math.exp(x) computese ** x, 


math.exp(@) # 7.6 
math.exp(1) # 2.718281828459645 (e) 


The function math.expm1(x) computese ** x - 1.When x is small, this gives significantly better precision than 


math.exp(x) - 1. 


math.expm1(@) # 0.0 
math.exp(1le-6) - 1 # 1.0000004999621837e-66 


math.expm1(1e-6) # 1.0000005000001665e-06 
# exact result # 1.000000500000166666708333341666... 


Section 9.4: Trigonometric Functions 


import math 


math.sin(a) # returns the sine of ‘a’ in radians 
# Out: @.8414709848078965 


math.cosh(b) # returns the inverse hyperbolic cosine of 'b' in radians 
# Out: 3.7621956910836314 


math.atan(math.pi) # returns the arc tangent of '‘pi' in radians 
# Out: 1.2626272556789115 


math.hypot(a, b) # returns the Euclidean norm, same as math.sqrt(a*a + bxb) 


# Out: 2.23606797749979 


Note that math.hypot(x, y) is also the length of the vector (or Euclidean distance) from the origin (8, ®) 
to the point (x, y). 


To compute the Euclidean distance between two points (x1, y1) & (x2, y2) you can use math.hypot as 
follows 


math.hypot(x2-x1, y2-y1) 
To convert from radians -> degrees and degrees -> radians respectively use math.degrees and math. radians 


math.degrees(a) 
# Out: 57.29577951308232 


math. radians(57 .29577951308232) 
# Out: 1.0 
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Section 9.5: Inplace Operations 


It is common within applications to need to have code like this: 


a=ati 
or 
a=ax*2 


at=1 
# and 
a *= 2 


Any mathematic operator can be used before the '=' character to make an inplace operation: 


e -= decrement the variable in place 

e += increment the variable in place 

e x= multiply the variable in place 

e /= divide the variable in place 

e //= floor divide the variable in place # Python 3 
e %= return the modulus of the variable in place 
**= raise to a power in place 


Other in place operators exist for the bitwise operators (*, | etc) 


Section 9.6: Subtraction 


ae beats 2: 

# Using the "-" operator: 

b-a #= 1 

import operator # contains 2 argument arithmetic functions 
operator.sub(b, a) #=7 


Possible combinations (builtin types): 


e int and int (gives an int) 

int and float (gives a float) 

e int and complex (gives a complex) 

e float and float (gives a float) 

float and complex (gives a complex) 
complex and complex (gives a complex) 


Section 9.7: Multiplication 


import operator 
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operator.mul(a, b) 


Possible combinations (builtin types): 


e int and int (gives an int) 

int and float (gives a float) 

e int and complex (gives a complex) 

e float and float (gives a float) 

float and complex (gives a complex) 
complex and complex (gives a complex) 


Note: The * operator is also used for repeated concatenation of strings, lists, and tuples: 


3 * 'ab' # = ‘ababab' 
Seem ai) eo (a lt, ely 19 Oe) 


Section 9.8: Logarithms 


By default, the math. 1og function calculates the logarithm of a number, base e. You can optionally specify a base as 


the second argument. 


import math 
import cmath 


math. 1log(5) # = 1.6094379124341003 
# optional base argument. Default is math.e 
math.log(5, math.e) # = 1.6094379124341663 


cmath.log(5) # = (1.6094379124341603+6]) 
math.log(100@, 10) # 3.@ (always returns float) 


cmath.log(1@00, 10) # (3467) 


Special variations of the math. log function exist for different bases. 


# Logarithm base e - 1 (higher precision for low values) 


math. 1log1p(5) # = 1.791759469228055 


# Logarithm base 2 
math. 1log2(8) # = 3.0 


# Logarithm base 10 


math.1o0g10( 180) # = 2.0 
cmath.1og10(100) # = (2407) 


Section 9.9: Modulus 


Like in many other languages, Python uses the % operator for calculating modulus. 


3% 4 # 3 
180 % 2 # @ 
6% 4 #2 


Or by using the operator module: 


import operator 


operator.mod(3 , 4) #3 
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operator.mod(1@ , 2) #@ 
operator.mod(6 , 4) Hee. 


You can also use negative numbers. 
yews 


# 
Oo =e # 
ee a ae # 


5 

-5 

=2 

If you need to find the result of integer division and modulus, you can use the divmod function as a shortcut: 


quotient, remainder = divmod(9, 4) 
# quotient = 2, remainder = 1 as 4 * 2+ 1 == 9 
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VIDEO: Python for Data 
Science and Machine 
Learning Bootcamp 


Learn how to use NumPy, Pandas, Seaborn, 
Matplotlib , Plotly, Scikit-Learn , Machine Learning, 
Tensorflow, and more! 


¥v Use Python for Data Science and Machine Learning 
v Use Spark for Big Data Analysis 

¥ Implement Machine Learning Algorithms 

¥ Learn to use NumPy for Numerical Data 

¥Y Learn to use Pandas for Data Analysis 

¥v Learn to use Matplotlib for Python Plotting 

v Learn to use Seaborn for statistical plots 

Vv Use Plotly for interactive dynamic visualizations 
Vv Use Scikit-Learn for Machine Learning Tasks 

v K-Means Clustering 

¥ Logistic Regression 

v Linear Regression 

¥ Random Forest and Decision Trees 

¥ Neural Networks 


v Support Vector Machines Ww re + re h T O ‘eo ‘eo v = 


Chapter 10: Bitwise Operators 


Bitwise operations alter binary strings at the bit level. These operations are incredibly basic and are directly 
supported by the processor. These few operations are necessary in working with device drivers, low-level graphics, 
cryptography, and network communications. This section provides useful knowledge and examples of Python's 
bitwise operators. 


Section 10.1: Bitwise NOT 


The ~ operator will flip all of the bits in the number. Since computers use signed number representations — most 
notably, the two's complement notation to encode negative binary numbers where negative numbers are written 
with a leading one (1) instead of a leading zero (0). 


This means that if you were using 8 bits to represent your two's-complement numbers, you would treat patterns 
from 8088 8808 to 8111 1111 to represent numbers from 0 to 127 and reserve 1xxx xxxx to represent negative 
numbers. 


Eight-bit two's-complement numbers 


Bits Unsigned Value Two's-complement Value 


0000 0000 O 0 
0000 0001 1 1 
0000 0010 2 2 
0111 1110 126 126 
01111111 127 127 
1000 0000 128 -128 
1000 0001 129 -127 
1000 0010 130 -126 
1111 1110 254 -2 
11111111 255 -1 


In essence, this means that whereas 1018 8118 has an unsigned value of 166 (arrived at by adding (128 * 1) + 
(64 * @) + (32 * 1) + (16 * 8) + (8 * 8) + (4% 1) + (2 * 1) + (1 * 8)), ithas a two's-complement value 
of -90 (arrived at by adding (128 * 1) - (64 * 8) - (32 * 1) - (16 * 8) - (8 * 8) - (4 * 1) - (2 * 1) - 
(1 * ®), and complementing the value). 


In this way, negative numbers range down to -128 (1800 28008). Zero (0) is represented as 8000 9800, and minus 
one (-1) as 1111 1111. 


In general, though, this means ~n = -n - 1, 


# @ = @beeee eeee 
~@ 

# Out: -1 

# -1 = @b1111 1111 


#1 = @beee6 8001 
~1 

# Out: -2 

# -2 = 1111 1118 


# 2 = @be000 0010 
2 


Goalkicker.com - Python® Notes for Professionals 65 


# Out: -3 
# -3 = 0b1111 1161 


# 123 = @b@111 1011 
~123 

# Out: -124 

# -124 = @b1000 6100 


Note, the overall effect of this operation when applied to positive numbers can be summarized: 
~n -> -|n+1| 

And then, when applied to negative numbers, the corresponding effect is: 
~-n -> |[n-1| 


The following examples illustrate this last rule... 


# -@ = @be8ee Bae0 

~-0 

# Out: -17 

# -1 = @b11171 1111 

# @ is the obvious exception to this rule, as -@ == @ always 


# -1 = @b100@ 6001 
~-1 

# Out: @ 

# @ = @beeee eeae 


# -2 = 6b1111 1118 
~-2 

# Out: 17 

#71 = @beee8 8001 


# -123 = @b1111 1011 
~-123 

# Out: 122 

# 122 = @b@111 1010 


Section 10.2: Bitwise XOR (Exclusive OR) 


The * operator will perform a binary XOR in which a binary 1 is copied if and only if it is the value of exactly one 
operand. Another way of stating this is that the result is 1 only if the operands are different. Examples include: 


# 60 = 6b111160 
# 3@ = @b@11110 
60) * 30 

# Out: 34 

# 34 = @b100010 


bin(6@ * 30) 
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# Out: 6b100016 


Section 10.3: Bitwise AND 


The & operator will perform a binary AND, where a bit is copied if it exists in both operands. That means: 


ee RR 

2-2 20 

R Lo L 

22-0 
mM 

~ 220 


# 66 = 0b111100 
# 30 = 0b@11116 
68 & 30 

# Out: 28 

# 28 = 6b11100 


bin(6@ & 30) 
# Out: @b11100 


Section 10.4: Bitwise OR 


The | operator will perform a binary "or," where a bit is copied if it exists in either operand. That means: 


| 
sa Au A ® 


# 60 = 6b111180 
# 3@ = @b@11110 
60 | 30 

# Out: 62 

# 62 = @b111110 


bin(6@ | 30) 
# Out: @b111110 


Section 10.5: Bitwise Left Shift 


The << operator will perform a bitwise "left shift," where the left operand's value is moved left by the number of bits 
given by the right operand. 


# 2 = @b10 

2 << 2 

# Out: 8 

# 8 = 6b1000 


bin(2 << 2) 
# Out: @b1000 


Performing a left bit shift of 1 is equivalent to multiplication by 2: 


Ti ees | 
# Out: 14 


Performing a left bit shift of n is equivalent to multiplication by 2**n: 
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Sh tats at) 
# Out: 48 


Section 10.6: Bitwise Right Shift 


The >> operator will perform a bitwise "right shift," where the left operand's value is moved right by the number of 


bits given by the right operand. 


# 8 = 6b1000 
8 >> 2 

# Out: 2 

# 2 = @b10 


bin(8 >> 2) 
# Out: @b10 


Performing a right bit shift of 1 is equivalent to integer division by 2: 


36 >> 1 
# Out: 18 


15S > 1 
# Out: 7 


Performing a right bit shift of n is equivalent to integer division by 2**n: 


48 >> 4 
# Oui 3 


Sie Beas 6 | 
FOULS aT 


Section 10.7: Inplace Operations 
All of the Bitwise operators (except ~) have their own in place versions 


a = 0be0e1 
&= @b010 
# a = 6beea0 


o 


= @bee1 
|= 90b010 
# a = @b@11 


a = 0be0e1 
a <<= 2 
# a = 0b100 


a = 0b100 
a >>= 2 
# a = @b@@7 


a = 0b101 


a 4= @b011 
# a = 0b110 
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Chapter 11: Boolean Operators 


Section 11.1: and and or are not guaranteed to return a 


boolean 


When you use or, it will either return the first value in the expression if it's true, else it will blindly return the second 


value. |.e. or is equivalent to: 


def or_(a, b): 
if a: 
return a 
else: 
return b 


For and, it will return its first value if it's false, else it returns the last value: 


def and_(a, b): 
if not a: 
return a 
else: 
return b 


Section 11.2: A simple example 


In Python you can compare a single element using two binary operators--one on either side: 


Dh Se 1A xc Sal AD 
print("x is near pi") 


In many (most?) programming languages, this would be evaluated in a way contrary to regular math: (3.14 < x) < 


3.142, but in Python it is treated like 3.14 < x and x < 3.142, just like most non-programmers would expect. 


Section 11.3: Short-circuit evaluation 


Python minimally evaluates Boolean expressions. 


>>> def true_func(): 
print("true_func()") 
return True 


>>> def false_func(): 
print("false_func()") 
return False 


>>> true_func() or false_func() 
true_func() 

True 

>>> false_func() or true_func() 
false_func() 

true_func() 

True 

>>> true_func() and false_func() 
true_func() 

false_func() 

False 

>>> false_func() and false_func() 
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false_func() 


False 


Section 11.4: and 


Evaluates to the second argument if and only if both of the arguments are truthy. Otherwise evaluates to the first 


falsey argument. 


x = True 
y = True 
Z = X and 
x = True 
y = False 
Z = X and 
x = False 
y = True 
Z = X and 
x = False 
y = False 
Z = X and 
x= 1 
y= 1 
Z = X and 
x = 0 
y 1 
Z = xX and 
x= 1 
y= 0 
Z = X and 
x = @ 
y= 0 
Z = xX and 


The 1's in the above example can be changed to any truthy value, and the 8's can be changed to any falsey value. 


y # 


True 


False 


False 


False 


XG 


so 


so 


so 


so 


Section 11.5: or 


Evaluates to the first truthy argument if either one of the arguments is truthy. If both arguments are falsey, 


@ (see 


@ (see 


@ (see 


evaluates to the second argument. 


x = True 
y = True 


Z=xory#z 


x = True 
y = False 


Z=xory#z 


x = False 
y = True 


Z=xory#z 


True 


True 


True 


‘and° 


above) 


above) 


above) 


and 
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Z =x or y # z = False 


xX = 1 
y=1 
Z =x or y #zZ =x, so z = 1, see ‘and’ and ‘or’ are not guaranteed to be a boolean 
x = 1 
y=@0 


Z =x or y #z =x, so z = 1 (see above) 


Z=xor y #z=y, so z = 1 (see above) 


<< 
® 


Z=xory#z=y, so z = @ (see above) 


The 1's in the above example can be changed to any truthy value, and the 8's can be changed to any falsey value. 


Section 11.6: not 
It returns the opposite of the following statement: 


x = True 
y = not x # y = False 


x = False 
y = not x # y = True 
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Chapter 12: Operator Precedence 


Python operators have a set order of precedence, which determines what operators are evaluated first in a 
potentially ambiguous expression. For instance, in the expression 3 * 2 + 7, first 3 is multiplied by 2, and then the 
result is added to 7, yielding 13. The expression is not evaluated the other way around, because * has a higher 
precedence than +. 


Below is a list of operators by precedence, and a brief description of what they (usually) do. 


Section 12.1: Simple Operator Precedence Examples in python 


Python follows PEMDAS rule. PEMDAS stands for Parentheses, Exponents, Multiplication and Division, and Addition 
and Subtraction. 


Example: 


Pe tly 10le Orn, ele 2, Si tos, 7) 
>>> a ** (b +c) # parentheses 


>>> a * b ** Cc # exponent: same as ‘a * (b ** c)° 

7776 

>>> a tb-*c/d =# multiplication / division: same as ‘a + (b * c/d)° 
4.142857142857142 


Extras: mathematical rules hold, but not always: 


>>> 300 / 300 * 200 

200.0 

>>> 300 x 200 / 300 

200.0 

>>> 1e300 / 1e300 * 1e200 
1e+200 

>>> 1e300 * 1e200 / 1e300 
inf 
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Chapter 13: Variable Scope and Binding 


Section 13.1: Nonlocal Variables 


Python 3.x Version = 3.0 


Python 3 added a new keyword called nonlocal. The nonlocal keyword adds a scope override to the inner scope. 
You can read all about it in PEP 3104. This is best illustrated with a couple of code examples. One of the most 
common examples is to create function that can increment: 


def counter(): 


num = @ 
def incrementer(): 
num += 1 


return num 
return incrementer 


If you try running this code, you will receive an UnboundLocalError because the num variable is referenced before 
it is assigned in the innermost function. Let's add nonlocal to the mix: 


def counter(): 
num = @ 
def incrementer(): 
nonlocal num 
num += 1 
return num 
return incrementer 


c = counter() 


CQ = 
c() # = 2 
COs =73 


Basically nonlocal will allow you to assign to variables in an outer scope, but not a global scope. So you can't use 
nonlocal in our counter function because then it would try to assign to a global scope. Give it a try and you will 
quickly get a SyntaxError. Instead you must use nonlocal in a nested function. 


(Note that the functionality presented here is better implemented using generators.) 


Section 13.2: Global Variables 


In Python, variables inside functions are considered local if and only if they appear in the left side of an assignment 
statement, or some other binding occurrence; otherwise such a binding is looked up in enclosing functions, up to 
the global scope. This is true even if the assignment statement is never executed. 


x = ‘Hi! 


def read_x(): 
print(x) # x is just referenced, therefore assumed global 


read_x() # prints Hi 


def read_y(): 
print(y) # here y is just referenced, therefore assumed global 
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read_y() # NameError: global name 'y' is not defined 


def read_y(): 
y = 'Hey' # y appears in an assignment, therefore it's local 
print(y) # will find the local y 


read_y() # prints Hey 


def read_x_local_fail(): 
if False: 
x = 'Hey'’ # x appears in an assignment, therefore it's local 
print (x) # will look for the _local_ z, which is not assigned, and will not be found 


read_x_local_fail() # UnboundLocalError: local variable 'x' referenced before assignment 


Normally, an assignment inside a scope will shadow any outer variables of the same name: 


def change_local_x(): 

x = 'Bye' 

print(x) 
change_local_x() # prints Bye 
print(x) # prints Hi 


Declaring a name global means that, for the rest of the scope, any assignments to the name will happen at the 
module's top level: 


x = ‘Hi! 


def change_global_x(): 
global x 
x = 'Bye' 
print(x) 


change_global_x() # prints Bye 
print(x) # prints Bye 


The global keyword means that assignments will happen at the module's top level, not at the program's top level. 
Other modules will still need the usual dotted access to variables within the module. 


To summarize: in order to know whether a variable x is local to a function, you should read the entire function: 


. if you've found global x, then x is a global variable 

. If you've found nonlocal x, then x belongs to an enclosing function, and is neither local nor global 
. If you've found x = 5o0rfor x in range(3) or some other binding, then x is a local variable 

. Otherwise x belongs to some enclosing scope (function scope, global scope, or builtins) 


BWN > 


Section 13.3: Local Variables 


If aname is bound inside a function, it is by default accessible only within the function: 
def foo(): 

a=5 

print(a) # ok 


print(a) # NameError: name ‘a' is not defined 
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Control flow constructs have no impact on the scope (with the exception of except), but accessing variable that was 
not assigned yet is an error: 


def foo(): 
if True: 
a=5 
print(a) # ok 


be=ng 
def bar(): 
if False: 


b= 5 
print(b) # UnboundLocalError: local variable 'b' referenced before assignment 


Common binding operations are assignments, for loops, and augmented assignments such as a += 5 


Section 13.4: The del command 


This command has several related yet distinct forms. 


del v 


If v is a variable, the command del v removes the variable from its scope. For example: 


x = 5 

print(x) # out: 5 

del x 

print(x) # NameError: name 'f' is not defined 


Note that del is a binding occurrence, which means that unless explicitly stated otherwise (using nonlocal 
or global), del v will make v local to the current scope. If you intend to delete v in an outer scope, use 
nonlocal vorglobal vinthe same scope of the del v statement. 


In all the following, the intention of a command is a default behavior but is not enforced by the language. A class 
might be written in a way that invalidates this intention. 


del v.name 
This command triggers a call to v.__delattr__(name). 


The intention is to make the attribute name unavailable. For example: 


class A: 
pass 


a= A() 

a.x = 7 

print(a.x) # out: 7 

del a.x 

print(a.x) # error: AttributeError: 'A' object has no attribute 'x' 


del v[item] 
This command triggers a call to v.__delitem__(item). 


The intention is that item will not belong in the mapping implemented by the object v. For example: 
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Xo eee dlncss lle seb) 23), 

del x['a'] 

print(x) # out: {'b': 2} 
print(x['a']) # error: KeyError: ‘a’ 


del v[a:b] 
This actually calls v._.delslice__(a, b). 


The intention is similar to the one described above, but with slices - ranges of items instead of a single item. For 
example: 


Xess (ION. die 2) 3) 4) 
del x[1:3] 
print(x) # out: [@, 3, 4] 


See also Garbage Collection#The del command. 


Section 13.5: Functions skip class scope when looking up 
names 


Classes have a local scope during definition, but functions inside the class do not use that scope when looking up 
names. Because lambdas are functions, and comprehensions are implemented using function scope, this can lead 
to some surprising behavior. 


a = ‘global’ 


class Fred: 
a = ‘class' # class scope 
= (a for i in range(1@)) # function scope 
= [a for i in range(10)] # function scope 
=a # class scope 
= lambda: a # function scope 
= lambda a=a: a # default argument uses class scope 


mnAOoaQaodnana 


@staticmethod # or @classmethod, or regular instance method 
def g(): # function scope 
return a 


print(Fred.a) # class 

print(next(Fred.b)) # global 

print(Fred.c[@]) # class in Python 2, global in Python 3 
print(Fred.d) # class 

print(Fred.e()) # global 

print(Fred.f()) # class 

print(Fred.g()) # global 


Users unfamiliar with how this scope works might expect b, c, and e to print class. 


From PEP 227: 


Names in class scope are not accessible. Names are resolved in the innermost enclosing function scope. 
If a class definition occurs in a chain of nested scopes, the resolution process skips class definitions. 


From Python's documentation on naming and binding: 
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The scope of names defined in a class block is limited to the class block; it does not extend to the code 
blocks of methods - this includes comprehensions and generator expressions since they are 
implemented using a function scope. This means that the following will fail: 


class A: 
a = 42 
b = list(a + i for i in range(1@)) 


This example uses references from this answer by Martijn Pieters, which contains more in depth analysis of this 
behavior. 


Section 13.6: Local vs Global Scope 


What are local and global scope? 
All Python variables which are accessible at some point in code are either in /ocal scope or in global scope. 


The explanation is that local scope includes all variables defined in the current function and global scope includes 
variables defined outside of the current function. 


foo = 1 # global 


def func(): 
bar = 2 # local 
print(foo) # prints variable foo from global scope 
print(bar) # prints variable bar from local scope 


One can inspect which variables are in which scope. Built-in functions locals() and globals() return the whole 
scopes as dictionaries. 


foo = 1 
def func(): 
bar = 2 


print(globals().keys()) # prints all variable names in global scope 
print(locals().keys()) # prints all variable names in local scope 


What happens with name clashes? 
foo = 1 


def func(): 
foo = 2 # creates a new variable foo in local scope, global foo is not affected 


print(foo) # prints 2 
# global variable foo still exists, unchanged: 


print(globals()['foo']) # prints 1 
print(locals()['foo']) # prints 2 


To modify a global variable, use keyword global: 


foo = 1 
def func(): 


global foo 
foo = 2 # this modifies the global foo, rather than creating a local variable 
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The scope is defined for the whole body of the function! 


What it means is that a variable will never be global for a half of the function and local afterwards, or vice-versa. 


foo 


def 


=| 


func(): 


# This function has a local variable foo, because it is defined down below. 


# So, foo is local from this point. Global foo is hidden. 


print(foo) # raises UnboundLocalError, because local foo is not yet initialized 


foo = 7 
print(foo) 


Likewise, the opposite: 


foo 


def 


= 1 


func(): 
# In this function, foo is a global variable from the beginning 


foo = 7 # global foo is modified 


print(foo) #7 
print(globals()['foo']) # 7 


global foo # this could be anywhere within the function 
print(foo) #7 


Functions within functions 


There may be many levels of functions nested within functions, but within any one function there is only one local 


scope for that function and the global scope. There are no intermediate scopes. 


foo 


def 


= 1 


faire 
bar = 1 


def f2(): 
baz = 2 
# here, foo is a global variable, baz is a local variable 
# bar is not in either scope 
print(locals().keys()) # ['baz'] 
print('bar' in locals()) # False 
print('bar' in globals()) # False 


def f3(): 
baz = 3 


print(bar) # bar from f1 is referenced so it enters local scope of f3 (closure) 


print(locals().keys()) # [‘'bar', ‘baz'] 
print('bar' in locals()) # True 
print('bar' in globals()) # False 


def f4(): 
bar = 4 # a new local bar which hides bar from local scope of f1 
baz = 4 
print(bar) 


print(locals().keys()) # ['bar', ‘baz'] 
print('bar' in locals()) # True 
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print('bar' in globals()) # False 


global Vs nonlocal (Python 3 only) 
Both these keywords are used to gain write access to variables which are not local to the current functions. 
The global keyword declares that a name should be treated as a global variable. 


foo = @ # global foo 


def f1(): 
foo = 1 # a new foo local in f1 


def f2(): 
foo = 2 # a new foo local in f2 


def f3(): 
foo = 3. # a new foo local in f3 
print(foo) #3 
foo = 30 # modifies local foo in f3 only 


def f4(): 
global foo 
print(foo) # @ 
foo = 100 # modifies global foo 


On the other hand, nonlocal (see Nonlocal Variables ), available in Python 3, takes a /ocal variable from an 
enclosing scope into the local scope of current function. 


From the Python documentation on nonlocal: 


The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest 
enclosing scope excluding globals. 


Python 3.x Version = 3.0 
def f1(): 


def f2(): 
foo = 2 # a new foo local in f2 


def f3(): 
nonlocal foo # foo from f2, which is the nearest enclosing scope 
print(foo) # 2 
foo = 20 # modifies foo from f2! 


Section 13.7: Binding Occurrence 


x= 5 
xX += 7 
for x in iterable: pass 


Each of the above statements is a binding occurrence - x become bound to the object denoted by 5. If this statement 


appears inside a function, then x will be function-local by default. See the "Syntax" section for a list of binding 
statements. 
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Chapter 14: Conditionals 


Conditional expressions, involving keywords such as if, elif, and else, provide Python programs with the ability to 
perform different actions depending on a boolean condition: True or False. This section covers the use of Python 
conditionals, boolean logic, and ternary statements. 


Section 14.1: Conditional Expression (or "The Ternary 
Operator”) 


The ternary operator is used for inline conditional expressions. It is best used in simple, concise operations that are 
easily read. 


e The order of the arguments is different from many other languages (such as C, Ruby, Java, etc.), which may 
lead to bugs when people unfamiliar with Python's "surprising" behaviour use it (they may reverse the order). 

e Some find it "unwieldy", since it goes contrary to the normal flow of thought (thinking of the condition first 
and then the effects). 


n= 5 


"Greater than 2" if n > 2 else "Smaller than or equal to 2" 
# Out: ‘Greater than 2' 


The result of this expression will be as it is read in English - if the conditional expression is True, then it will evaluate 
to the expression on the left side, otherwise, the right side. 


Ternary operations can also be nested, as here: 


n= 5 
"Hello" if n > 10 else "Goodbye" if n > 5 else "Good day" 


They also provide a method of including conditionals in lambda functions. 


Section 14.2: if, elif, and else 


In Python you can define a series of conditionals using if for the first one, elif for the rest, up until the final 
(optional) else for anything not caught by the other conditionals. 


number = 5 

if number > 2: 
print("Number is bigger than 2.") 

elif number < 2: # Optional clause (you can have multiple elifs) 
print("Number is smaller than 2.") 


else: # Optional clause (you can only have one else) 
print("Number is 2.") 


Outputs Number is bigger than 2 


Using else if instead of elif will trigger a syntax error and is not allowed. 


Section 14.3: Truth Values 


The following values are considered falsey, in that they evaluate to False when applied to a boolean operator. 
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e None 


e False 
e 8, or any numerical value equivalent to zero, for example @L, 8.2, @j 
e Empty sequences: ''', "", (), [] 


e Empty mappings: {} 
e User-defined types where the __bool__ or __len__ methods return 0 or False 


All other values in Python evaluate to True. 


Note: A common mistake is to simply check for the Falseness of an operation which returns different Falsey values 
where the difference matters. For example, using if foo() rather than the more explicit if foo() is None 


Section 14.4: Boolean Logic Expressions 


Boolean logic expressions, in addition to evaluating to True or False, return the value that was interpreted as True 
or False. It is Pythonic way to represent logic that might otherwise require an if-else test. 


And operator 


The and operator evaluates all expressions and returns the last expression if all expressions evaluate to True. 
Otherwise it returns the first value that evaluates to False: 


>>> 1 and 2 
2Z 


>>> 1 and @ 
7) 


>>> 1 and "Hello World" 
"Hello World" 


>>> and "Pancakes" 


Or operator 


The or operator evaluates the expressions left to right and returns the first value that evaluates to True or the last 
value (if none are True). 


Soo a Or 2 
7 


>>> None or 1 
1 


>>> 8 or [] 


[] 


Lazy evaluation 


When you use this approach, remember that the evaluation is lazy. Expressions that are not required to be 
evaluated to determine the result are not evaluated. For example: 
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>>> def print_me(): 
print('I am here!') 

>>> @ and print_me() 

i) 


In the above example, print_me is never executed because Python can determine the entire expression is False 
when it encounters the @ (False). Keep this in mind if print_me needs to execute to serve your program logic. 


Testing for multiple conditions 


A common mistake when checking for multiple conditions is to apply the logic incorrectly. 


This example is trying to check if two variables are each greater than 2. The statement is evaluated as- if (a) and 


(b > 2). This produces an unexpected result because bool(a) evaluates as True when a is not zero. 


sso. = | 

s>> b = 6 
>>> if a and b > 2: 
print('yes') 

. else: 
print('no') 


yes 
Each variable needs to be compared separately. 


>>> if a > 2 and b > 2: 
print('yes' ) 
. else: 
print('no') 


no 


Another, similar, mistake is made when checking if a variable is one of multiple values. The statement in this 


example is evaluated as- if (a == 3) or (4) or (6). This produces an unexpected result because bool(4) and 


bool(6) each evaluate to True 


sso) qe = | 
>>> if a == 3 or 4 or 6: 
print('yes' ) 
. else: 
print('no') 


yes 


Again each comparison must be made separately 


>>>. if a ——. 3 OF a:-— 4 OM a) —— 6) 
print('yes' ) 
. else: 
print('‘no') 
no 


Using the in operator is the canonical way to write this. 
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So> af aan (3, 4, 6) 
print('yes' ) 
. else: 
print('‘no') 
no 


Section 14.5: Using the cmp function to get the comparison 
result of two objects 


Python 2 includes a cmp function which allows you to determine if one object is less than, equal to, or greater than 
another object. This function can be used to pick a choice out of a list based on one of those three options. 


Suppose you need to print ‘greater than' ifx > y, ‘less than' ifx < yand ‘equal' ifx == y. 


['equal', ‘greater than', ‘less than', ][cmp(x,y)] 


# x,y = 1,1 output: ‘equal’ 
# x,y = 1,2 output: ‘less than' 
# x,y = 2,1 output: ‘greater than' 


cmp(x, y) returns the following values 


Comparison Result 


x<y -1 
X==y 0 
X>y 1 


This function is removed on Python 3. You can use the cmp_to_key( func) helper function located in functools in 
Python 3 to convert old comparison functions to key functions. 


Section 14.6: Else statement 


if condition: 
body 
else: 
body 


The else statement will execute it's body only if preceding conditional statements all evaluate to False. 


if True: 
print "It is true!" 
else: 
print "This won't get printed.. 


# Output: It is true! 


if False: 

print "This won't get printed... 
else: 

print "It is false!" 


# Output: It is false! 
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Section 14.7: Testing if an object is None and assigning it 
You'll often want to assign something to an object if it is None, indicating it has not been assigned. We'll use aDate. 
The simplest way to do this is to use the is None test. 


if aDate is None: 
aDate=datetime.date.today() 


(Note that it is more Pythonic to say is None instead of == None.) 


But this can be optimized slightly by exploiting the notion that not None will evaluate to True in a boolean 
expression. The following code is equivalent: 


if not aDate: 
aDate=datetime.date.today() 


But there is a more Pythonic way. The following code is also equivalent: 
aDate=aDate or datetime.date.today() 


This does a Short Circuit evaluation. If aDate is initialized and is not None, then it gets assigned to itself with no net 
effect. If it is None, then the datetime.date.today() gets assigned to aDate 


Section 14.8: If statement 


if condition: 
body 


The if statements checks the condition. If it evaluates to True, it executes the body of the if statement. If it 
evaluates to False, it skips the body. 


if True: 
print "It is true!" 
>> It is true! 


if False: 
print "This won't get printed..." 


The condition can be any valid expression: 


if 2 +2 == 
print "I know math!" 
>> I know math! 
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VIDEO: Machine 
Spe eutle A-Z: Hands-On 
Python In Data Science 


Learn to create Machine Learning Algorithms in 
Python trom two Data Science experts. Code 
templates included. 


¥ Master Machine Learning on Python 

¥ Have a great intuition of many Machine Learning models 

Y¥ Make accurate predictions 

Y¥ Make powerful analysis 

Y¥ Make robust Machine Learning models 

Y Create strong added value to your business 

¥ Use Machine Learning for personal purpose 

Y Handle specific topics like Reinforcement Learning, NLP and Deep Learning 
¥ Handle advanced techniques like Dimensionality Reduction 

¥ Know which Machine Learning model to choose for each type of problem 
Y Build an army of powerful Machine Learning models and know how to combine them to solve any 
problem 


Watch Today — 


Chapter 15: Comparisons 


Parameter Details 
X First item to be compared 
y Second item to be compared 


Section 15.1: Chain Comparisons 


You can compare multiple items with multiple comparison operators with chain comparison. For example 
Xx >yrzZ 

is just a short form of: 

x > y and y > z 

This will evaluate to True only if both comparisons are True. 

The general form is 
a OP b OP c OP dd... 


Where OP represents one of the multiple comparison operations you can use, and the letters represent arbitrary 
valid expressions. 


Note that @ != 1 != @ evaluates to True, even though 8 != @ is False. Unlike the common mathematical 
notation in which x != y != z means that x, y and z have different values. Chaining == operations has 
the natural meaning in most cases, since equality is generally transitive. 


Style 


There is no theoretical limit on how many items and comparison operations you use as long you have proper 
syntax: 


de <2 SO). <<: 100) =) 24 


The above returns True if each comparison returns True. However, using convoluted chaining is not a good style. A 
good chaining will be "directional", not more complicated than 


i exXe ce A ayn 


Side effects 


As soon as one comparison returns False, the expression evaluates immediately to False, skipping all remaining 
comparisons. 


Note that the expression exp ina > exp > b will be evaluated only once, whereas in the case of 
a > exp and exp > b 
exp will be computed twice if a > exp is true. 
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a a 


Section 15.2: Comparison by ‘is vs == 

A common pitfall is confusing the equality comparison operators is and ==. 
a == b compares the value of a and b. 

a is b will compare the identities of a and b. 

To illustrate: 


= 'Python is fun!’ 
= 'Python is fun!' 
== b # returns True 
is b # returns False 


oo Oo ® 


[ie 253. 4 25] 

=a # b references a 

b # True 

b # True 

= Eube | # b now references a copy of a 
8) # True 

b # False [!!] 


oo ono D9 OD DW 
H 
72) 


Basically, is can be thought of as shorthand for id(a) == id(b). 


Beyond this, there are quirks of the run-time environment that further complicate things. Short strings and small 
integers will return True when compared with is, due to the Python machine attempting to use less memory for 
identical objects. 


= 'short' 
‘short’ 
=5 

ao 

is b # True 
is d # True 


oo 200 ® 


But longer strings and larger integers will be stored separately. 


is b # False 
is d # False 


a = 'not so short' 
b = 'not so short' 
c = 1000 

d = 1000 

a 

c 


You should use is to test for None: 


if myvar is not None: 
# not None 
pass 
if myvar is None: 
# None 
pass 


Ause of is is to test for a “sentinel” (i.e. a unique object). 


sentinel = object() 
def myfunc(var=sentinel) : 
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if var is sentinel: 
# value wasn’t provided 
pass 


else: 
# value was provided 


pass 


Section 15.3: Greater than or less than 


These operators compare two types of values, they're the less than and greater than operators. For numbers this 


simply compares the numerical values to see which is larger: 


oad 
# True 
2 <4 
# False 
i <4 
# True 


For strings they will compare lexicographically, which is similar to alphabetical order but not quite the same. 


"alpha" < "beta" 


# True 

"gamma" > "beta" 
# True 

"gamma" < "OMEGA" 
# False 


In these comparisons, lowercase letters are considered ‘greater than' uppercase, which is why "gamma" < "OMEGA" 
is false. If they were all uppercase it would return the expected alphabetical ordering result: 


"GAMMA" < "OMEGA" 
# True 
Each type defines it's calculation with the < and > operators differently, so you should investigate what the 


operators mean with a given type before using it. 


Section 15.4: Not equal to 


This returns True if x and y are not equal and otherwise returns False. 


12 != 1 

# True 

As ee 

# True 

cays a= ale 
# False 
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Section 15.5: Equal To 


x< 
I 
iT] 
< 


This expression evaluates if x and y are the same value and returns the result as a boolean value. Generally both 
type and value need to match, so the int 12 is not the same as the string '12'. 


12 == 12 

# True 

12 == 1 

# False 

"42" == '12' 

# True 

‘spam’ == 'spam' 
# True 

‘spam’ == 'spam 
# False 

"12' == 12 

# False 


Note that each type has to define a function that will be used to evaluate if two values are the same. For builtin 
types these functions behave as you'd expect, and just evaluate things based on being the same value. However 
custom types could define equality testing as whatever they'd like, including always returning True or always 
returning False. 


Section 15.6: Comparing Objects 


In order to compare the equality of custom classes, you can override == and != by defining _.eq__ and __ne__ 
methods. You can also override __1t__ (<), __le__ (<=), __gt__ (>), and __ge__ (>). Note that you only need to 
override two comparison methods, and Python can handle the rest (== is the same as not < and not >, etc.) 


class Foo(object): 
def __init__(self, item): 
self.my_item = item 
def __eq__(self, other): 
return self.my_item == other.my_item 


iT] ll 
717 
oo 
[omme) 
—~ 
aw 
wwe 


b # True 
!= b # False 
b # False 


oo 9 O D 
= 
iT] 


Note that this simple comparison assumes that other (the object being compared to) is the same object type. 
Comparing to another type will throw an error: 


class Bar(object): 
def __init__(self, item): 
self.other_item = item 
def __eq__(self, other): 


return self.other_item == other.other_item 
def __ne__(self, other): 
return self.other_item != other.other_item 
c = Bar(5) 
a==c # throws AttributeError: 'Foo' object has no attribute ‘other_item' 
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Checking isinstance() or similar will help prevent this (if desired). 
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Chapter 16: Loops 


Parameter Details 
boolean expression expression that can be evaluated in a boolean context, e.g. x < 10 


variable variable name for the current element from the iterable 
iterable anything that implements iterations 


As one of the most basic functions in programming, loops are an important piece to nearly every programming 
language. Loops enable developers to set certain portions of their code to repeat through a number of loops which 
are referred to as iterations. This topic covers using multiple types of loops and applications of loops in Python. 


Section 16.1: Break and Continue in Loops 


break statement 


When a break statement executes inside a loop, control flow "breaks" out of the loop immediately: 


i=@ 
while i < 7: 
print(i) 
if i == 
print("Breaking from loop") 
break 
i += 1 


The loop conditional will not be evaluated after the break statement is executed. Note that break statements are 
only allowed inside loops, syntactically. A break statement inside a function cannot be used to terminate loops that 
called that function. 


Executing the following prints every digit until number 4 when the break statement is met and the loop stops: 


DWDHRWNF © 


reaking from loop 


break statements can also be used inside for loops, the other looping construct provided by Python: 
for i: in (Gs 1, 2, 37 4): 

print(i) 

if i == 


break 


Executing this loop now prints: 


Note that 3 and 4 are not printed since the loop has ended. 
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If a loop has an else clause, it does not execute when the loop is terminated through a break statement. 


continue statement 


A continue statement will skip to the next iteration of the loop bypassing the rest of the current block but 
continuing the loop. As with break, continue can only appear inside loops: 


for ain (@, 1), 2; 
if ie 2 Of a: 
continue 
print(i) 


3 


’ 


APOE: 
AR 


aw —- © 


Note that 2 and 4 aren't printed, this is because continue goes to the next iteration instead of continuing on to 
print(i) wheni == 2ori == 4. 
Nested Loops 


break and continue only operate on a single level of loop. The following example will only break out of the inner 


for loop, not the outer while loop: 


while True: 
for i in range(1,5): 
if i == 2: 
break # Will only break out of the inner loop! 


Python doesn't have the ability to break out of multiple levels of loop at once -- if this behavior is desired, 
refactoring one or more loops into a function and replacing break with return may be the way to go. 


Use return from within a function as a break 


The return statement exits from a function, without executing the code that comes after it. 


If you have a loop inside a function, using return from inside that loop is equivalent to having a break as the rest of 


the code of the loop is not executed (note that any code after the loop is not executed either): 


def break_loop(): 
for i in range(1, 5): 
if (i == 2): 
return(i) 
print(i) 
return(5) 


If you have nested loops, the return statement will break all loops: 


def break_all(): 
for j in range(1, 5): 
for i in range(1,4): 
if ixj == 6: 
return(i) 
print(i*j) 


will output: 
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# 1x7 

# 1%2 

# 1%3 

# 1%4 

# 27 

# 22 

return because 2%3 = 6, the remaining iterations of both loops are not executed 


*e RN BR WD 


Section 16.2: For loops 


for loops iterate over a collection of items, such as list or dict, and run a block of code with each element from 
the collection. 


for an [es i, 2S Ae 
print(i) 


The above for loop iterates over a list of numbers. 


Each iteration sets the value of i to the next element of the list. So first it will be ®, then 1, then 2, etc. The output 
will be as follow: 


BWNF © 


range is a function that returns a series of numbers under an iterable form, thus it can be used in for loops: 


for i in range(5): 
print(i) 


gives the exact same result as the first for loop. Note that 5 is not printed as the range here is the first five 
numbers counting from 8. 


Iterable objects and iterators 


for loop can iterate on any iterable object which is an object which defines a ___getitem__ or a ___iter__ function. 


The __iter__ function returns an iterator, which is an object with a next function that is used to access the next 
element of the iterable. 


Section 16.3: Iterating over lists 


To iterate through a list you can use for: 


for x in ['one', ‘two', ‘three’, ‘four']: 
print(x) 


This will print out the elements of the list: 


one 
two 
three 
four 
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The range function generates numbers which are also often used in a for loop. 


for x in range(1, 6): 
print(x) 


The result will be a special range sequence type in python >=3 and a list in python <=2. Both can be looped through 
using the for loop. 


OBWNFH 


If you want to loop though both the elements of a list and have an index for the elements as well, you can use 
Python's enumerate function: 


for index, item in enumerate(['one', ‘two', ‘three’, ‘four']): 


print (index, :', item) 


enumerate will generate tuples, which are unpacked into index (an integer) and item (the actual value from the list). 
The above loop will print 


(Gre ones )) 
Ginn ee etwor) 
(Gye = athnees) 
(Soe fours: 


Iterate over a list with value manipulation using map and lambda, i.e. apply lambda function on each element in the 
list: 


X = map(lambda e : e.upper(), ['one', ‘two', 'three', ‘four']) 


print (x) 
Output: 
['ONE', 'TWO', 'THREE', ‘FOUR'] # Python 2.x 


NB: in Python 3.x map returns an iterator instead of a list so you in case you need a list you have to cast the result 
print(list(x)) 


Section 16.4: Loops with an "else” clause 


The for and while compound statements (loops) can optionally have an else clause (in practice, this usage is fairly 
rare). 


The else clause only executes after a for loop terminates by iterating to completion, or after a while loop 
terminates by its conditional expression becoming false. 


for i in range(3): 
print(i) 

else: 
print('done') 


i=@0 
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while i < 3: 
print(i) 
i += 1 

else: 
print('done') 


output: 


The else clause does not execute if the loop terminates some other way (through a break statement or by raising 


an exception): 


for i in range(2): 
print(i) 
if i == 1: 
break 
else: 
print('done' ) 


output: 


Most other programming languages lack this optional else clause of loops. The use of the keyword else in 


particular is often considered confusing. 


The original concept for such a clause dates back to Donald Knuth and the meaning of the else keyword becomes 


clear if we rewrite a loop in terms of if statements and goto statements from earlier days before structured 
programming or from a lower-level assembly language. 


For example: 


while loop_condition(): 


if break_condition(): 
break 


is equivalent to: 


# pseudocode 


<<start>>: 
if loop_condition(): 


if break_condition(): 
goto <<end>> 


goto <<start>> 
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<<end>>: 


These remain equivalent if we attach an else clause to each of them. 


For example: 


while loop_condition(): 


if break_condition(): 
break 


else: 
print('done' ) 


is equivalent to: 


# pseudocode 


<<start>>: 
if loop_condition(): 


if break_condition(): 
goto <<end>> 


goto <<start>> 
else: 
print('done' ) 


<<end>>: 


A for loop with an else clause can be understood the same way. Conceptually, there is a loop condition that 
remains True as long as the iterable object or sequence still has some remaining elements. 


Why would one use this strange construct? 
The main use case for the for. ..else construct is a concise implementation of search as for instance: 


Ay hile 22, She 
for i ina: 
if type(i) is not int: 
print(i) 
break 
else: 
print("no exception" ) 


To make the else in this construct less confusing one can think of it as "if not break" or "if not found". 


Some discussions on this can be found in [Python-ideas] Summary of for...else threads, Why does python use ‘else’ 
after for and while loops?, and Else Clauses on Loop Statements 


Section 16.5: The Pass Statement 
pass is a null statement for when a statement is required by Python syntax (such as within the body of a for or 
while loop), but no action is required or desired by the programmer. This can be useful as a placeholder for code 


that is yet to be written. 


for x in range(1@): 
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pass #we don't want to do anything, or are not ready to do anything here, so we'll pass 


In this example, nothing will happen. The for loop will complete without error, but no commands or code will be 


actioned. pass allows us to run our code successfully without having all commands and action fully implemented. 


Similarly, pass can be used in while loops, as well as in selections and function definitions etc. 


while x == 
pass 


Section 16.6: Iterating over dictionaries 
Considering the following dictionary: 

Carta ay be a ie er) 
To iterate through its keys, you can use: 


for key in d: 
print(key) 


Output: 


This is equivalent to: 


for key in d.keys(): 
print(key) 


or in Python 2: 


for key in d.iterkeys(): 
print(key) 


To iterate through its values, use: 


for value in d.values(): 
print(value) 


Output: 


To iterate through its keys and values, use: 


for key, value in d.items(): 
print(key, "::", value) 


Output: 
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Note that in Python 2, .keys(), .values() and .items() return a list object. If you simply need to iterate through 
the result, you can use the equivalent .iterkeys(), .itervalues() and .iteritems(). 


The difference between .keys() and .iterkeys(), .values() and .itervalues(), .items() and .iteritems() is 
that the iter* methods are generators. Thus, the elements within the dictionary are yielded one by one as they are 
evaluated. When a list object is returned, all of the elements are packed into a list and then returned for further 
evaluation. 


Note also that in Python 3, Order of items printed in the above manner does not follow any order. 


Section 16.7: The "half loop” do-while 


Unlike other languages, Python doesn't have a do-until or a do-while construct (this will allow code to be executed 
once before the condition is tested). However, you can combine awhile True with a break to achieve the same 
purpose. 


a = 10 

while True: 
a= a-1 
print(a) 
if a<7: 


break 
print('Done.') 


This will print: 


Section 16.8: Looping and Unpacking 
If you want to loop over a list of tuples for example: 
roto iicronennoyny Se [NCTE ye lh tO Ne er, ee I, BZ, 
instead of doing something like this: 
for item in collection: 
i1 = item[@] 
i2 = item[1] 
i3 = item[2] 
# logic 
or something like this: 


for item in collection: 
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i1, 12, i3 = item 
# logic 


You can simply do this: 


for i1, i2, i3 in collection: 
# logic 


This will also work for most types of iterables, not just tuples. 


Section 16.9: Iterating different portion of a list with different 


step size 


Suppose you have a long list of elements and you are only interested in every other element of the list. Perhaps you 
only want to examine the first or last elements, or a specific range of entries in your list. Python has strong indexing 


built-in capabilities. Here are some examples of how to achieve these scenarios. 
Here's a simple list that will be used throughout the examples: 

lst = ['alpha', ‘bravo', ‘charlie’, ‘delta’, '‘echo'] 

Iteration over the whole list 


To iterate over each element in the list, a for loop like below can be used: 


for s in lst: 
print s[:1] # print the first letter 


The for loop assigns s for each element of 1st. This will print: 


o0oaQadqnd»ow® 


Often you need both the element and the index of that element. The enumerate keyword performs that task. 


for idx, s in enumerate(lst): 
print("%s has an index of %d" % (s, idx)) 


The index idx will start with zero and increment for each iteration, while the s will contain the element being 
processed. The previous snippet will output: 


alpha has an index of 0 
bravo has an index of 1 
charlie has an index of 2 
delta has an index of 3 
echo has an index of 4 


Iterate over sub-list 


If we want to iterate over a range (remembering that Python uses zero-based indexing), use the range keyword. 
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for i in range(2,4): 
print("Ist at %d contains %s" % (i, lst[i])) 


This would output: 


lst at 2 contains charlie 
lst at 3 contains delta 
The list may also be sliced. The following slice notation goes from element at index 1 to the end with a step of 2. 
The two for loops give the same result. 
for s in lIst[1::2]: 
print(s) 
for i in range(1, len(l1st), 2): 
print(lst[i]) 


The above snippet outputs: 


bravo 
delta 


Indexing and slicing is a topic of its own. 


Section 16.10: While Loop 


A while loop will cause the loop statements to be executed until the loop condition is falsey. The following code will 


execute the loop statements a total of 4 times. 


i=@ 

while i < 4: 
#loop statements 
i=ittl 


While the above loop can easily be translated into a more elegant for loop, while loops are useful for checking if 
some condition has been met. The following loop will continue to execute until myObject is ready. 


myObject = anObject() 
while myObject.isNotReady(): 
myObject.tryToGetReady( ) 


while loops can also run without a condition by using numbers (complex or real) or True: 


import cmath 


complex_num = cmath.sqrt(-1) 
while complex_num: # You can also replace complex_num with any number, True or a value of any 


type 


print(complex_num) # Prints 1j forever 


If the condition is always true the while loop will run forever (infinite loop) if it is not terminated by a break or return 


statement or an exception. 


while True: 
print "Infinite loop" 
100 
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# Infinite loop 
# Infinite loop 
# Infinite loop 
Hosa 
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Chapter 17: Arrays 


Parameter Details 

b Represents signed integer of size 1 byte 

B Represents unsigned integer of size 1 byte 

c Represents character of size 1 byte 

u Represents unicode character of size 2 bytes 
h Represents signed integer of size 2 bytes 

H Represents unsigned integer of size 2 bytes 


i Represents signed integer of size 2 bytes 

I Represents unsigned integer of size 2 bytes 
w Represents unicode character of size 4 bytes 
1 Represents signed integer of size 4 bytes 

L Represents unsigned integer of size 4 bytes 
f Represents floating point of size 4 bytes 

d Represents floating point of size 8 bytes 


"Arrays" in Python are not the arrays in conventional programming languages like C and Java, but closer to lists. A 
list can be a collection of either homogeneous or heterogeneous elements, and may contain ints, strings or other 
lists. 


Section 17.1: Access individual elements through indexes 
Individual elements can be accessed through indexes. Python arrays are zero-indexed. Here is an example: 
my_array = array('i', [1,2,3,4,5]) 

print(my_array[1]) 

#2 

print(my_array[2]) 

# 3 


print(my_array[®@]) 
# 71 


Section 17.2: Basic Introduction to Arrays 


An array is a data structure that stores values of same data type. In Python, this is the main difference between 
arrays and lists. 


While python lists can contain values corresponding to different data types, arrays in python can only contain 
values corresponding to same data type. In this tutorial, we will understand the Python arrays with few examples. 


If you are new to Python, get started with the Python Introduction article. 


To use arrays in python language, you need to import the standard array module. This is because array is not a 
fundamental data type like strings, integer etc. Here is how you can import array module in python : 


from array import * 


Once you have imported the array module, you can declare an array. Here is how you do it: 


arrayIdentifierName = array(typecode, [Initializers]) 
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In the declaration above, arrayIdentifierName is the name of array, typecode lets python know the type of array 
and Initializers are the values with which array is initialized. 


Typecodes are the codes that are used to define the type of array values or the type of array. The table in the 
parameters section shows the possible values you can use when declaring an array and it's type. 


Here is a real world example of python array declaration : 

my_array = array(‘'i',[1,2,3,4]) 

In the example above, typecode used is i. This typecode represents signed integer whose size is 2 bytes. 
Here is a simple example of an array containing 5 integers 


from array import * 

my_array = array('i', [1,2,3,4,5]) 
for i in my_array: 

print(i) 

# 71 

# 2 

# 3 

#4 

#5 


Section 17.3: Append any value to the array using append() 
method 


my_array = array('i', [1,2,3,4,5]) 

my_array.append(6) 

He vallalcaly, (ecdicyean live oe AS DOs) 

Note that the value 6 was appended to the existing array values. 


Section 17.4: Insert value in an array using insert() method 


We can use the insert() method to insert a value at any index of the array. Here is an example : 


ivyeathay = akhav( i) 9) lille, 374, 610) 
my_array.insert(@,9) 
farray('i , [@, 1, 2, 3, 4, 5]) 


In the above example, the value 0 was inserted at index 0. Note that the first argument is the index while second 


argument is the value. 


Section 17.5: Extend python array using extendQ method 


A python array can be extended with more than one value using extend() method. Here is an example: 


my_array = array('i', [1,2,3,4,5]) 
my_extnd_array = array(‘i', [7,8,9,10]) 
my_array.extend(my_extnd_array) 

Le CNR NA th, MM yen Ch, We Cle a TY, 


We see that the array my_array was extended with values from my_extnd_array. 
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Section 17.6: Add items from list into array using fromlistQ 
method 


Here is an example: 

My2dnGay =sarreyiod elie, 4,51) 
C= dai] 

my_array.fromlist(c) 

Le eNereya ake ly, By ein Zo Oy A EH, 


So we see that the values 11,12 and 13 were added from list c to my_array. 


Section 17.7: Remove any array element using removeQ) 
method 


Here is an example: 

myoankay = array(. i, [1,2,3,4,5])) 
my_array.remove(4) 

cz ERA abe Will, 2) Se) ip) 


We see that the element 4 was removed from the array. 


Section 17.8: Remove last array element using popQ method 
pop removes the last element from the array. Here is an example : 

my_array = array('i', [1,2,3,4,5]) 

my_array.pop() 

ue Glelreya( ak hil, Py Sie), 


So we see that the last element (5) was popped out of array. 


Section 17.9: Fetch any element through its index using index() 
method 

index() returns first index of the matching value. Remember that arrays are zero-indexed. 

my_array = array('i', [1,2,3,4,5]) 

print (my_array.index(5) ) 

#5 

my_array = array('i', [1,2,3,3,5]) 


print (my_array.index(3) ) 
# 3 


Note in that second example that only one index was returned, even though the value exists twice in the array 
Section 17.10: Reverse a python array using reverseQ method 
The reverse() method does what the name says it will do - reverses the array. Here is an example: 


fivearray = array( a°, [1,2,3,4,51) 
my_array.reverse() 
array tee Soe wc ay)) 
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Section 17.11: Get array buffer information through 
buffer_infoQ method 


This method provides you the array buffer start address in memory and number of elements in array. Here is an 
example: 


mysarray = apray( 2, [1,2,3,4,5)) 
my_array.buffer_info() 
(33881712, 5) 


Section 17.12: Check for number of occurrences of an element 
using count) method 


count() will return the number of times and element appears in an array. In the following example we see that the 
value 3 occurs twice. 


ny airay = array i, [2.3.98 5]) 
my_array.count(3) 
# 2 


Section 17.13: Convert array to string using tostring() method 


tostring() converts the array to a string. 


my_char_array = array('c', ['g',‘e',‘e','k']) 
# array('c’', ‘geek') 
print(my_char_array.tostring()) 

# geek 


Section 17.14: Convert ples Ass to a python list with same 
elements using tolistQ met 


When you need a Python list object, you can utilize the tolist() method to convert your array to a list. 
my-ahray = array(’i', [1,2,3,4,51) 


c = my_array.tolist() 
# [1, 2, 3, 4, $] 


Section 17.15: Append a string to char array using fromstringQ 
method 


You are able to append a String to a character array using fromstring() 
my_char_array = array(‘c', ['g',‘e',‘e','‘k']) 
my_char_array. poneemanat SSUH RE) 


print (my_char_array) 
#array('c', ‘geekstuff') 
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Chapter 18: Multidimensional arrays 


Section 18.1: Lists in lists 


A good way to visualize a 2d array is as a list of lists. Something like this: 
Ist=[[1,2,3], [4,5,6], [7,8,9]] 


here the outer list 1st has three things in it. each of those things is another list: The first one is: [1,2,3], the second 
one is: [4,5,6] and the third one is: [7, 8,9]. You can access these lists the same way you would access another 


other element of a list, like this: 


print (lst[@]) 
cAonaines itn 2, sul 


print (lst[1]) 
#output: [4, 5, 6] 


print (lst[2]) 
#output: [7, 8, 9] 


You can then access the different elements in each of those lists the same way: 


print (lst[@][0]) 
#output: 17 


print (lst[@][1]) 
#output: 2 


Here the first number inside the [] brackets means get the list in that position. In the above example we used the 
number ® to mean get the list in the Oth position which is [1, 2,3]. The second set of [] brackets means get the 
item in that position from the inner list. In this case we used both 8 and 1 the Oth position in the list we got is the 


number 1 and in the 1st position it is 2 
You can also set values inside these lists the same way: 
Ist[@]=[10, 11,12] 


Now the list is [[10,11,12],[4,5,6],[7,8,9]]. In this example we changed the whole first list to be a completely 


new list. 
Ist[1][2]=15 


Now the list is [[10,11,12],[4,5,15],[7,8,9]]. In this example we changed a single element inside of one of the 
inner lists. First we went into the list at position 1 and changed the element within it at position 2, which was 6 now 


it's 15. 
Section 18.2: Lists in lists in lists in.. 


This behaviour can be extended. Here is a 3-dimensional array: 


tie is l2i22 2S lise 2 (Soll Mi 2i e222 222 223i 2s zoo, 23S i St, oie, 
Sieiple2 ieee 32 |lsetooe.es3ilail 
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As is probably obvious, this gets a bit hard to read. Use backslashes to break up the different dimensions: 


LMA SIP 22 12a Melis 233%] /FN 
(212122 13 224 222-923) - i 2od2325 233 KN 
Sti s12 73131) [20138225323)| (330332333) Mi] 


By nesting the lists like this, you can extend to arbitrarily high dimensions. 
Accessing is similar to 2D arrays: 


print(myarray) 
print(myarray[1]) 
print(myarray[2][1]) 
print(myarray[1][@][2]) 
etc. 


And editing is also similar: 


myarray[1]=new_n-1_d_list 

myarray[2][1]=new_n-2_d_list 

myarray[1][@][2]=new_n-3_d_list #or a single number if you're dealing with 3D arrays 
etc. 
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Chapter 19: Dictionary 


Parameter Details 
key The desired key to lookup 
value The value to set or return 


Section 19.1: Introduction to Dictionary 


A dictionary is an example of a key value store also known as Mapping in Python. It allows you to store and retrieve 
elements by referencing a key. As dictionaries are referenced by key, they have very fast lookups. As they are 
primarily used for referencing items by key, they are not sorted. 


creating a dict 
Dictionaries can be initiated in many ways: 


literal syntax 


d= {} # empty dict 
d = {'key': ‘value’ } # dict with initial values 


Python 3.x Version = 3.5 

# Also unpacking one or multiple dictionaries with the literal syntax is possible 
# makes a shallow copy of otherdict 

d = {**otherdict} 


# also updates the shallow copy with the contents of the yetanotherdict. 
d = {xxotherdict, **yetanotherdict} 


dict comprehension 
d = {k:v for k,v in [(‘key', ‘value',)]} 


see also: Comprehensions 


built-in class: dict() 


d = dict() # empty dict 

d = dict(key='value' ) # explicit keyword arguments 

d = dict([('key', ‘value')]) # passing in a list of key/value pairs 

# make a shallow copy of another dict (only possible if keys are only strings!) 


d = dict(**otherdict) 
modifying a dict 


To add items to a dictionary, simply create a new key with a value: 
d['newkey'] = 42 
It also possible to add list and dictionary as value: 


d['new_list'] 
d['new_dict' ] 


[1, 25 3] 
{'nested_dict': 1} 


To delete an item, delete the key from the dictionary: 


del d['newkey' ] 
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Section 19.2: Avoiding KeyError Exceptions 


One common pitfall when using dictionaries is to access a non-existent key. This typically results in a KeyError 
exception 


mydict = {} 

mydict[ ‘not there’ ] 

Traceback (most recent call last): 
File "<stdin>", line 1, in <module> 

KeyError: ‘not there' 


One way to avoid key errors is to use the dict .get method, which allows you to specify a default value to return in 
the case of an absent key. 


value = mydict.get(key, default_value) 


Which returns mydict [key] if it exists, but otherwise returns default_value. Note that this doesn't add key to 
mydict. So if you want to retain that key value pair, you should use mydict.setdefault(key, default_value), 
which does store the key value pair. 


mydict = {} 

print(mydict) 

# {} 

print(mydict.get("foo", "bar")) 
# bar 

print (mydict) 

# {} 
print(mydict.setdefault("foo", "bar")) 
# bar 

print (mydict) 

Had CEOOw fe bala): 


An alternative way to deal with the problem is catching the exception 


try: 

value = mydict[key] 
except KeyError: 

value = default_value 


You could also check if the key is in the dictionary. 
if key in mydict: 
value = mydict[key] 


else: 
value = default_value 


Do note, however, that in multi-threaded environments it is possible for the key to be removed from the dictionary 
after you check, creating a race condition where the exception can still be thrown. 


Another option is to use a subclass of dict, collections.defaultdict, that has a default_factory to create new entries in 
the dict when given a new_key. 


Section 19.3: Iterating Over a Dictionary 


If you use a dictionary as an iterator (e.g. in a for statement), it traverses the keys of the dictionary. For example: 
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Ol re als le 2a ves 
for key in d: 

print(key, d[key]) 
#c 3 
#b2 
#ail 


The same is true when used in a comprehension 
print([key for key in d]) 
HolleCuneDs anal 


Python 3.x Version = 3.0 


The items() method can be used to loop over both the key and value simultaneously: 


for key, value in d.items(): 
print(key, value) 

#c 3 

#b2 

#al 


While the values() method can be used to iterate over only the values, as would be expected: 


for key, value in d.values(): 
print(key, value) 
# 3 
# 2 
#7 


Python 2.x Version = 2.2 


Here, the methods keys(), values() and items() return lists, and there are the three extra methods iterkeys() 
itervalues() and iteritems() to return iterators. 


Section 19.4: Dictionary with default values 
Available in the standard library as defaultdict 


from collections import defaultdict 


d = defaultdict (int) 


d['key' ] #0 
d['key'] = 5 
d['key' ] oP 6) 


d = defaultdict(lambda: ‘empty’ ) 


d['key'] # ‘empty’ 
d['key'] = ‘full’ 
d['key' ] # ‘full’ 


[*] Alternatively, if you must use the built-in dict class, using dict.setdefault() will allow you to create a default 
whenever you access a key that did not exist before: 


>>> d = {} 

{} 

>>> d.setdefault('Another_key', []).append("This worked!") 
>>> d 
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{'Another_key': ['This worked! ' ]} 


Keep in mind that if you have many values to add, dict.setdefault() will create a new instance of the initial value 
(in this example a []) every time it's called - which may create unnecessary workloads. 


[*] Python Cookbook, 3rd edition, by David Beazley and Brian K. Jones (O'Reilly). Copyright 2013 David Beazley and Brian 
Jones, 978-1-449-34037-7. 


Section 19.5: Merging dictionaries 

Consider the following dictionaries: 

>>> fish = {'name': "Nemo", ‘hands': "fins", ‘special’: "gills"} 
>>> dog = {'name': "Clifford", ‘hands': "paws", ‘color': "red"} 


Python 3.5+ 


>>> fishdog = {**xfish, **dog} 
>>> fishdog 
{'hands': 'paws', ‘color’: 'red', ‘name’: ‘Clifford’, ‘special’: 'gills'} 


As this example demonstrates, duplicate keys map to their lattermost value (for example "Clifford" overrides 
"Nemo’"). 


Python 3.3+ 


>>> from collections import ChainMap 
>>> dict(ChainMap(fish, dog)) 
{'hands': 'fins', ‘color’: 'red', ‘special’: 'gills', ‘name’: 'Nemo'} 


With this technique the foremost value takes precedence for a given key rather than the last ("Clifford" is thrown 
out in favor of "Nemo"). 


Python 2.x, 3.x 


>>> from itertools import chain 
>>> dict(chain(fish.items(), dog.items()) ) 
{hands » “paws’, “collor’: “red’, “name's “Clififord’; “specials “gills” }: 


This uses the lattermost value, as with the «*-based technique for merging ("Clifford" overrides "Nemo’"). 


>>> fish.update(dog) 
>>> fish 
{color = :red'; ‘hands’ = “paws; ‘name = “Gillififord,, “special: “giillis™} 


dict .update uses the latter dict to overwrite the previous one. 


Section 19.6: Accessing keys and values 


When working with dictionaries, it's often necessary to access all the keys and values in the dictionary, either ina 
for loop, a list comprehension, or just as a plain list. 


Given a dictionary like: 


mydict = { 


ait) elie 
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You can get a list of keys using the keys() method: 

print (mydict.keys() ) 

# Python2: ['a’', ‘b'] 

# Python3: dict_keys(['b', ‘a']) 

If instead you want a list of values, use the values() method: 
print(mydict.values() ) 


A@PychOnZc weil GeeZe} 
# Python3: dict_values(['2', '1']) 


If you want to work with both the key and its corresponding value, you can use the items() method: 
print(mydict.items()) 
A lMdioe (ey oe ot, ee 


# Python3: dict_items([i(°b', °2°), (‘a , 1 )]) 


NOTE: Because a dict is unsorted, keys(), values(), and items() have no sort order. Use sort(), sorted(), or an 
OrderedDict if you care about the order that these methods return. 


Python 2/3 Difference: In Python 3, these methods return special iterable objects, not lists, and are the equivalent 
of the Python 2 iterkeys(), itervalues(), and iteritems() methods. These objects can be used like lists for the 
most part, though there are some differences. See PEP 3106 for more details. 


Section 19.7: Accessing values of a dictionary 


dictionary = {"Hello": 1234, "World": 5678} 
print(dictionary["Hello"]) 


The above code will print 1234. 


The string “Hello” in this example is called a key. It is used to lookup a value in the dict by placing the key in 
square brackets. 


The number 1234 is seen after the respective colon in the dict definition. This is called the va/ue that "Hello" maps 
to in this dict. 


Looking up a value like this with a key that does not exist will raise a KeyError exception, halting execution if 
uncaught. If we want to access a value without risking a KeyError, we can use the dictionary.get method. By 
default if the key does not exist, the method will return None. We can pass it a second value to return instead of 
None in the event of a failed lookup. 


w = dictionary.get( "whatever" ) 
x = dictionary.get("whatever", "“nuh-uh") 


In this example w will get the value None and x will get the value “nuh-uh". 


Section 19.8: Creating a dictionary 


Rules for creating a dictionary: 
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e Every key must be unique (otherwise it will be overridden) 
e Every key must be hashable (can use the hash function to hash it; otherwise TypeError will be thrown) 
e There is no particular order for the keys. 


# Creating and populating it with values 
stock = {'eggs': 5, ‘milk’: 2} 


# Or creating an empty dictionary 
dictionary = {} 


# And populating it after 
dictionary[‘eggs'] = 5 
dictionary[‘milk'] = 2 


# Values can also be lists 
mydict = {'a': [1, 2, 3], ‘b': ['one', ‘two', ‘three’ ]} 


# Use list.append() method to add new elements to the values list 
mydict[‘'a'].append(4) Hoo idee file 2) ore A Die sOner 6 stWOr matinee: at 
mydict['b'].append('four') # => {'a': [1, 2, 3, 4], ‘b': ['one', ‘two', ‘three’, ‘four']} 


# We can also create a dictionary using a list of two-items tuples 
iterable = [('eggs', 5), (‘milk', 2)] 
dictionary = dict(iterables) 


# Or using keyword argument: 
dictionary = dict(eggs=5, milk=2) 


# Another way will be to use the dict.fromkeys: 
dictionary = dict.fromkeys((milk, eggs)) # => {'milk': None, ‘eggs': None} 
dictionary = dict.fromkeys((milk, eggs), (2, 5)) # => {'milk': 2, ‘eggs’: 5} 


Section 19.9: Creating an ordered dictionary 


You can create an ordered dictionary which will follow a determined order when iterating over the keys in the 
dictionary. 


Use OrderedDict from the collections module. This will always return the dictionary elements in the original 
insertion order when iterated over. 


from collections import OrderedDict 


d = OrderedDict() 
dli*farnst, |= 1 
d['second'] = 2 
d['third'] = 3 
d['last'] = 4 


# Outputs “first 7.) -second 2°. third 3. last 4™ 


for key in d: 
print(key, d[key]) 


Section 19.10: Unpacking dictionaries using the ** operator 


You can use the ** keyword argument unpacking operator to deliver the key-value pairs in a dictionary into a 
function's arguments. A simplified example from the official documentation: 
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>>> def parrot(voltage, state, action): 


print("This parrot wouldn't", action, end=' ') 
print("if you put", voltage, "volts through it.", end=' ') 
print("E's", state, "!") 
>>> d = {"voltage": "four million", "state": "bleedin' demised", "action": "VOOM"} 


>>> parrot(**d) 

This parrot wouldn't VOOM if you put four million volts through it. E's bleedin’ demised ! 
As of Python 3.5 you can also use this syntax to merge an arbitrary number of dict objects. 

>>> fish = {'name': "Nemo", '‘hands': "fins", ‘special’: "gills"} 

>>> dog = {'name': "Clifford", ‘hands’: "paws", ‘color': "red"} 

>>> fishdog = {**xfish, **dog} 

>>> fishdog 


{'hands': 'paws', ‘color’: 'red', ‘name’: ‘Clifford’, ‘special’: 'gills'} 


As this example demonstrates, duplicate keys map to their lattermost value (for example "Clifford" overrides 
"Nemo'"). 


Section 19.11: The trailing comma 
Like lists and tuples, you can include a trailing comma in your dictionary. 


role = {"By day": "A typical programmer", 
"By night": "Still a typical programmer", } 


PEP 8 dictates that you should leave a space between the trailing comma and the closing brace. 


Section 19.12: The dict() constructor 


The dict() constructor can be used to create dictionaries from keyword arguments, or from a single iterable of 
key-value pairs, or from a single dictionary and keyword arguments. 


dict (a=lnib=2,..c=3)) See ap ef Poy a tas Bey 
oben Cal? eh Oe eI (is MID) ere Gl eS 2h ee ey Peas (ai) 
dict [l(a dl) b=25c=3)) eee ape May a eh. Ui ee 
Grote auc ie eulby sa2:) c= 3)) ce ev Ey S “ifs elo) A Vag eps 


Section 19.13: Dictionaries Example 


Dictionaries map keys to values. 


car = {} 

car["wheels"] = 4 

ear l| color’ |=" Red” 
car["model"] = "Corvette" 


Dictionary values can be accessed by their keys. 


print "Little " + car["color"] + + car["model"] + "!" 
# This would print out "Little Red Corvette!" 


Dictionaries can also be created in a JSON style: 
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car = {"wheels": 4, "color": "Red", "model": "Corvette" } 
Dictionary values can be iterated over: 


for key in car: 


print key + ": " + car[key] 


# wheels: 4 
# color: Red 
# model: Corvette 


Section 19.14: All combinations of dictionary values 


options = { 
Ug lane Foy! Il 
Ene [Gee 28) 3G) 


Given a dictionary such as the one shown above, where there is a list representing a set of values to explore for the 
corresponding key. Suppose you want to explore "x"="a" with “y"=1@, then "x"="a" with"y"=18, and so on until 
you have explored all possible combinations. 


You can create a list that returns all such combinations of values using the following code. 


import itertools 


options = { 
WEES lela abel 
nya [Ole 20,4830))) 


keys = options.keys() 

values = (options[key] for key in keys) 

combinations = [dict(zip(keys, combination)) for combination in itertools.product(*values) ] 
print combinations 


This gives us the following list stored in the variable combinations: 


lie are eee Oi 
fies Xeae er Dcemeean ieee aT 
Ae Gu oe tele pe WE RNP Re 
excl cal parler Ne 
ee. al yy OP Bete, 
EX De enV FY | 
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VIDEO: Machine ~ 
Learning, Data Science 
and Deep Learning with 
Python 


Complete hands-on machine learning tutorial with 
data science, Tensorflow, artificial intelligence, 
and neural networks 


Y Build artificial neural networks with Tensorflow and Keras 

Y Classify images, data, and sentiments using deep learning 

¥ Make predictions using linear regression, polynomial regression, and multivariate regression 
Y Data Visualization with MatPlotLib and Seaborn 

¥ Implement machine learning at massive scale with Apache Spark's MLLib 

v Understand reinforcement learning - and how to build a Pac-Man bot 

Y Classify data using K-Means clustering, Support Vector Machines (SVM), KNN, Decision Trees, 
Naive Bayes, and PCA 

Vv Use train/test and K-Fold cross validation to choose and tune your models 

Y¥ Build a movie recommender system using item-based and user-based collaborative filtering 


Watch Today — 


Chapter 20: List 


The Python List is a general data structure widely used in Python programs. They are found in other languages, 
often referred to as dynamic arrays. They are both mutable and a sequence data type that allows them to be indexed 
and sliced. The list can contain different types of objects, including other list objects. 


Section 20.1: List methods and supported operators 


Starting with a given list a: 


a = [1, 


2, 3, 4, 5] 


1. append(value) — appends a new element to the end of the list. 


e+ O29 O9 Oo 


Append values 6, 7, and 7 to the list 


.append(6) 
.append(7) 


append(7) 
AS Wi. Bo ste 2 Aye Gs Hy 7A 


Append another list 

= (8, 9] 

append(b) 

Ee Wn Pa Sly i a On ah, ey fhe OL 


Append an element of a different type, as list elements do not need to have the same type 


my_string = "hello world" 


a 
# 


.append(my_string) 


al I 2a S 4a O67, Gee See whellonworld.)] 


Note that the append() method only appends one new element to the end of the list. If you append a list to 
another list, the list that you append becomes a single element at the end of the first list. 


# Appending a list to another list 
pilin pine ce ee (le ae tA 

b = [8, 9] 

a.append(b) 

cee ip 2, ei, Zh ee Ge He oA IMS. SUL 
a[8] 

# Returns: [8,9] 


2. extend(enumerable) — extends the list by appending elements from another enumerable. 


® 


fe) 


ee ei a es On Parl 
lea 2, 16 


Extend list by appending all elements from b 


.extend(b) 


ENS Ili, 2 Shy Ch, Op Ze The Gh, le 112) 


# Extend list with elements from a non-list enumerable: 


* 


.extend(range(3) ) 


Ge ty 2 Gy Zh Oy Oy Wa Ue By Oy Wyo Ne 2 


Lists can also be concatenated with the + operator. Note that this does not modify any of the original lists: 
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a= a2) Se el tie Feb 
Poa [25 8) 4) <6 68 72 1798095 TO] 


3. index(value, [startIndex]) - gets the index of the first occurrence of the input value. If the input value is 
not in the list a ValueError exception is raised. If a second argument is provided, the search is started at that 
specified index. 


a.index(7) 
# Returns: 6 


a.index(49) # ValueError, because 49 is not in a. 


a.index(7, 7) 
# Returns: 7 


a.index(7, 8) # ValueError, because there is no 7 starting at index 8 


4, insert(index, value) —inserts value just before the specified index. Thus after the insertion the new 
element occupies position index. 


a.insert(@, 0) # insert @ at position @ 
a.insert(2, 5) # insert 5 at position 2 
“ele I, Un Sy 2a <i tip Oy Gp Te Coes Oe AO 


5. pop([index]) —- removes and returns the item at index. With no argument it removes and returns the last 
element of the list. 


a.pop(2) 

# Returns: 5 

ies I dy Zp Si 2 ee to V4, 2, Ge AIH 
a.pop(8) 

# Returns: 7 

ff Gs ANG. te en Bi, Che i, on ip se, NG HT 


# With no argument: 


a.pop() 
# Returns: 10 
cece Uh ip 2a sie Ce Oy. Oy Wy Ady 


6. remove(value) — removes the first occurrence of the specified value. If the provided value cannot be found, a 
ValueError is raised. 


a.remove(@) 

a.remove(9) 

cele Wife Be Ste Zi, dye (an 7h teh 
a.remove(10) 

# ValueError, because 1@ is not in a 


7. reverse() —reverses the list in-place and returns None. 


a.reverse() 
UM Ghe ts. Hy On Sp 2h ely An tl 


There are also other ways of reversing a list. 
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8. count(value) — counts the number of occurrences of some value in the list. 


a.count(7) 
# Returns: 2 


9. sort() —sorts the list in numerical and lexicographical order and returns None. 


a.sort() 
cee) Gb lie 2 sine Chey 6, 74, HI 
# Sorts the list in numerical order 


Lists can also be reversed when sorted using the reverse=True flag in the sort() method. 


a.sort(reverse=True) 
feel St IG Th Oe. Oe Ch ie Be. IL 


If you want to sort by attributes of items, you can use the key keyword argument: 


import datetime 


class Person(object): 
def __init__(self, name, birthday, height): 
self.name = name 
self.birthday = birthday 
self.height = height 


def __repr__(self): 
return self .name 


1 = [Person("John Cena", datetime.date(1992, 9, 12), 175), 
Person("Chuck Norris", datetime.date(1990, 8, 28), 180), 
Person("Jon Skeet", datetime.date(1991, 7, 6), 185)] 


l.sort(key=lambda item: item.name) 
1: [Chuck Norris, John Cena, Jon Skeet] 


ES 


1.sort(key=lambda item: item.birthday) 
# 1: [Chuck Norris, Jon Skeet, John Cena] 


a 


.sort(key=lambda item: item.height) 
1: [John Cena, Chuck Norris, Jon Skeet] 


ESS 


In case of list of dicts the concept is the same: 


import datetime 

1 = [{'name':'John Cena', ‘birthday’: datetime.date(1992, 9, 12), 'height': 175}, 
{'name': 'Chuck Norris', ‘birthday': datetime.date(1998, 8, 28), 'height': 180}, 
{'name': ‘Jon Skeet', ‘birthday’: datetime.date(1991, 7, 6), ‘height': 185}] 


1.sort(key=lambda item: item[ ‘name’ ]) 
# 1: [Chuck Norris, John Cena, Jon Skeet] 


1.sort(key=lambda item: item[ ‘birthday’ ]) 
# 1: [Chuck Norris, Jon Skeet, John Cena] 


1.sort(key=lambda item: item[ ‘height’ ]) 
# 1: [John Cena, Chuck Norris, Jon Skeet] 
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Sort by sub dict: 
import datetime 


1 = [{'name':'John Cena', ‘birthday’: datetime.date(1992, 9, 12),'size': {'height': 175, 
‘weight’: 100}}, 
‘Chuck Norris', 'birthday': datetime.date(1990, 8, 28),'size' : {'height': 180, 
weight': 90}}, 


name': ‘Jon Skeet', 'birthday': datetime.date(1991, 7, 6), ‘size': {'height': 185, 
weight': 110}}] 


1.sort(key=lambda item: item['size']['height' ]) 
#1: [John Cena, Chuck Norris, Jon Skeet] 


Better way to sort using attrgetter and itemgetter 


Lists can also be sorted using attrgetter and itemgetter functions from the operator module. These can help 
improve readability and reusability. Here are some examples, 


from operator import itemgetter,attrgetter 

people = [{'name':'chandan', 'age' :20, 'salary' :2000}, 
{'name':'chetan','age':18,'salary' :5000}, 
{'name':'guru','age':30,'salary' :3000}] 

by_age = itemgetter('age' ) 

by_salary = itemgetter('salary' ) 


people.sort(key=by_age) #in-place sorting by age 
people.sort(key=by_salary) #in-place sorting by salary 


itemgetter can also be given an index. This is helpful if you want to sort based on indices of a tuple. 


list_of_tuples = [(1,2), (3,4), (5,8)] 
list_of_tuples.sort(key=itemgetter(1)) 
print(list_of_tuples) #/(5, @), (1, 2), (3, 4)] 


Use the attrgetter if you want to sort by attributes of an object, 


persons = [Person("John Cena", datetime.date(1992, 9, 12), 175), 
Person("Chuck Norris", datetime.date(1990, 8, 28), 180), 


Person("Jon Skeet", datetime.date(1991, 7, 6), 185)] #reusing Person class from above 
example 


person.sort(key=attrgetter('name')) #sort by name 
by_birthday = attrgetter( ‘birthday’ ) 
person.sort(key=by_birthday) #sort by birthday 


10. clear() —removes all items from the list 


a.clear() 
#a={] 


11. Replication — multiplying an existing list by an integer will produce a larger list consisting of that many copies 
of the original. This can be useful for example for list initialization: 


b = ["blah"] * 3 
#b = ["blah", "blah", "blah"] 
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Ze ile Sie Sie tly ip On Up Sy We th Sip Oy tl ey ll 


Take care doing this if your list contains references to objects (eg a list of lists), see Common Pitfalls - List 
multiplication and common references. 


12. Element deletion — it is possible to delete multiple elements in the list using the del keyword and slice 
notation: 


a = list(range(19)) 
del a[::2] 

peep (ily el, Sy of, Ul 
del a[-1] 

ae) sh (hil, Sh Bp 74 
del a[:] 

#a=[] 


13. Copying 


The default assignment "=" assigns a reference of the original list to the new name. That is, the original name 
and new name are both pointing to the same list object. Changes made through any of them will be reflected 
in another. This is often not what you intended. 


b=a 
a.append(6) 
2 (8 Wi, Be ste 2 Ae 6h] 


If you want to create a copy of the list you have below options. 
You can slice it: 
new_list = old_list[:] 
You can use the built in list() function: 
new_list = list(old_list) 
You can use generic copy.copy(): 


import copy 
new_list = copy.copy(old_list) #inserts references to the objects found in the original. 


This is a little slower than list() because it has to find out the datatype of old_list first. 


If the list contains objects and you want to copy them as well, use generic copy.deepcopy(): 


import copy 
new_list = copy.deepcopy(old_list) #inserts copies of the objects found in the original. 


Obviously the slowest and most memory-needing method, but sometimes unavoidable. 


Python 3.x Version = 3.0 
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copy() — Returns a shallow copy of the list 


aa = a.copy() 
Had — ai 2,4 Ol 


Section 20.2: Accessing list values 

Python lists are zero-indexed, and act like arrays in other languages. 

ist = [1 2, 3, 4) 

Ist[0] #17 

Ist[1] #2 
Attempting to access an index outside the bounds of the list will raise an IndexError. 


Ist[4] # IndexError: list index out of range 


Negative indices are interpreted as counting from the end of the list. 
Ist[-1] #4 


lst[-2] #3 
Ist[-5] # IndexError: list index out of range 


This is functionally equivalent to 
Ist[len(1lst)-1] # 4 
Lists allow to use slice notation as 1st[ start :end:step]. The output of the slice notation is a new list containing 


elements from index start to end-1. If options are omitted start defaults to beginning of list, end to end of list and 
step to 1: 


Ist[1:] # (2, 3, 4] 

Ist[:3] ce (lila a Gill 

Ist[::2] hide 3] 

1lst[::-1] en [Ae Gree2 oe 1a) 

lst[-1:0:-1] # [4, 3, 2] 

Ist[5:8] # [] since starting index is greater than length of Ist, returns empty list 
1lst[1:10] # [2, 3, 4] same as omitting ending index 


With this in mind, you can print a reversed version of the list by calling 
1lst[::-1] Te Nek ei Oe at 


When using step lengths of negative amounts, the starting index has to be greater than the ending index otherwise 
the result will be an empty list. 


Ist[3:1:-1] # [4, 3] 


Using negative step indices are equivalent to the following code: 


reversed(1lst)[@:2] #@=1 -1 
#2 = 3 -1 


The indices used are 1 less than those used in negative indexing and are reversed. 
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Advanced slicing 


When lists are sliced the __getitem__() method of the list object is called, with a slice object. Python has a builtin 
slice method to generate slice objects. We can use this to store a slice and reuse it later like so, 


data = ‘chandan purohit 22 2000' #assuming data fields of fixed length 
name_slice = slice(@,19) 

age_slice = slice(19,21) 

salary_slice = slice(22,None) 


#now we can have more readable slices 
print(data[name_slice]) #chandan purohit 
print(data[age_slice]) #'22’ 
print(data[salary_slice]) #'2000' 


This can be of great use by providing slicing functionality to our objects by overriding __getitem__ in our class. 


Section 20.3: Checking if list is empty 


The emptiness of a list is associated to the boolean False, so you don't have to check len(1st) == 98, but just 1st 
ornot Ist 


Ist = [] 
if not lst: 
print("list is empty") 


# Output: list is empty 


Section 20.4: iterating over a list 
Python supports using a for loop directly on a list: 


my_list = ['foo', ‘bar', ‘baz'] 
for item in my_list: 
print (item) 


# Output: foo 
# Output: bar 
# Output: baz 


You can also get the position of each item at the same time: 


for (index, item) in enumerate(my_list) : 
print('The item in position {} is: {}'.format(index, item)) 


# Output: The item in position @ is: foo 
# Output: The item in position 1 is: bar 
# Output: The item in position 2 is: baz 


The other way of iterating a list based on the index value: 


for i in range(@, len(my_list)): 
print(my_list[i]) 

#output: 

>>> 

foo 

bar 
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baz 
Note that changing items in a list while iterating on it may have unexpected results: 


for item in my_list: 
if item == ‘foo': 
del my_list[@] 
print (item) 


# Output: foo 
# Output: baz 


In this last example, we deleted the first item at the first iteration, but that caused bar to be skipped. 


Section 20.5: Checking whether an item is in a list 
Python makes it very simple to check whether an item is in a list. Simply use the in operator. 


Ist = ['test', ‘twest', ‘tweast', ‘treast'] 


‘test' in lst 
# Out: True 


‘toast' in lst 
# Out: False 


Note: the in operator on sets is asymptotically faster than on lists. If you need to use it many times on 
potentially large lists, you may want to convert your list to a set, and test the presence of elements on 


the set. 


slst = set(lst) 
‘test' in slst 
# Out: True 


Section 20.6: Any and All 


You can use all1() to determine if all the values in an iterable evaluate to True 


nums = [1, 1, 8, 1] 
all(nums) 

# False 

eharseao( ar, ibe, ede] 
all(chars) 

# True 


Likewise, any() determines if one or more values in an iterable evaluate to True 


nums = [1, 1, @, 1] 

any(nums) 

# True 

vals = [None, None, None, False] 
any(vals) 

# False 


While this example uses a list, it is important to note these built-ins work with any iterable, including generators. 
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Weil et [ile 22, Shy 4] 

any(val > 12 for val in vals) 

# False 

any((val * 2) > 6 for val in vals) 
# True 


Section 20.7: Reversing list elements 


You can use the reversed function which returns an iterator to the reversed list: 


In [3]: rev = reversed(numbers) 


In [4]: rev 
Out (4 She 76-64 oe 48 25 | 


Note that the list "numbers" remains unchanged by this operation, and remains in the same order it was originally. 


To reverse in place, you can also use the reverse method. 


You can also reverse a list (actually obtaining a copy, the original list is unaffected) by using the slicing syntax, 


setting the third argument (the step) as -1: 


In [1]: numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9] 


In [2]: numbers[::-1] 
Ouc(Al Sree 7 6-9 oe 4 S| 


Section 20.8: Concatenate and Merge lists 


1. The simplest way to concatenate list1 and list2: 


merged = list1 + list2 


2. zip returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument 


sequences or iterables: 


WaDioe 'a3'] 
boi 'b3'] 


alist 
blist 


ian 
ao ® 
=o 


for a, b in zip(alist, blist): 
print(a, b) 


# Output: 
# al bl 
# a2 b2 
# a3 b3 


If the lists have different lengths then the result will include only as many elements as the shortest one: 


alist pret. ees Seer 

bidicte=e [bie wb2e es boi mb4) 4 

for a, b in zip(alist, blist): 
print(a, b) 


# Output: 
# al bil 
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# a2 b2 
# a3 b3 


alist = [] 
len(list(zip(alist, blist))) 


# Output: 
# 0 


For padding lists of unequal length to the longest one with Nones use itertools.zip_longest 
(itertools.izip_longest in Python 2) 


alas te=— [isa aee asi 
blist = ['b1'] 
CHaiSt— (cll aC 2umuCou mC Aull| 


for a,b,c in itertools.zip_longest(alist, blist, clist): 
print(a, b, c) 


# Output: 

# al bi cl 

# a2 None c2 
# a3 None c3 
# None None c4 


3. Insert to a specific index values: 


aliiste= 9 [1235 yZe 8 ezaikas  wabcul| 

alist.insert(3, [2009]) 

print("Final List :", alist) 
Output: 


Final List : [123, ‘xyz', ‘zara', 2009, ‘abc'] 


Section 20.9: Length of a list 

Use len() to get the one-dimensional length of a list. 

len(['one', ‘two']) # returns 2 

len(['one', [2, 3], ‘four']) # returns 3, not 4 

len() also works on strings, dictionaries, and other data structures similar to lists. 
Note that len() is a built-in function, not a method of a list object. 


Also note that the cost of len() is 0(1), meaning it will take the same amount of time to get the length of a list 
regardless of its length. 


Section 20.10: Remove duplicate values in list 


Removing duplicate values in a list can be done by converting the list to a set (that is an unordered collection of 
distinct objects). If a list data structure is needed, then the set can be converted back to a list using the function 
list(): 
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names = ["aixk", "duke", "edik", "tofp", "duke"] 


list(set(names) ) 
# Out: ['duke', '‘tofp', ‘aixk', ‘edik'] 


Note that by converting a list to a set the original ordering is lost. 


To preserve the order of the list one can use an OrderedDict 


import collections 
>>> collections.OrderedDict.fromkeys(names) .keys() 
AOuts || aixk; “dukes edika, )  tonp 4] 


Section 20.11: Comparison of lists 


It's possible to compare lists and other sequences lexicographically using comparison operators. Both operands 


must be of the same type. 


[1, 10, 100] < [2, 10, 100] 

# True, because 71 < 2 

[1, 10, 100] < [1, 10, 100] 

# False, because the lists are equal 
[1, 10, 100] <= [1, 10, 100] 

# True, because the lists are equal 
[1, 10, 100] < [1, 10, 101] 

# True, because 100 < 101 

[1, 10, 100] < [@, 10, 100] 

# False, because @ < 17 


If one of the lists is contained at the start of the other, the shortest list wins. 


areal < hi ess eel 
# True 


Section 20.12: Accessing values in nested list 


Starting with a three-dimensional list: 


alasts= "(ol [4,21), (esa hiss, 718.9, 10 [ize i, 14 ))1) 
Accessing items in the list: 


print(alist[@][@][1]) 


#2 
#Accesses second element in the first list in the first list 


print(alist[1][1][2]) 


#10 
#Accesses the third element in the second list in the second list 


Performing support operations: 


alist[@][9@].append(11) 

print(alist[@][@][2]) 

#11 

#Appends 11 to the end of the first list in the first list 
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Using nested for loops to print the list: 


for row in alist: #0ne way to loop through nested lists 
for col in row: 
print(col) 
CAL ey TE 
#[3, 4] 
CARop Op, All 
#[8, 9, 10] 
#[12, 13, 14] 


Note that this operation can be used in a list comprehension or even as a generator to produce efficiencies, e.g.: 


[col for row in alist for col in row] 
CAN 2, Ug MS il, WS Te, ZA, ksh, Si UO, We Wek Wea 


Not all items in the outer lists have to be lists themselves: 


alist[1].insert(2, 15) 
#Inserts 15 into the third position in the second list 


Another way to use nested for loops. The other way is better but I've needed to use this on occasion: 


for row in range(len(alist)): #4 less Pythonic way to loop through lists 
for col in range(len(alist[row])): 
print(alist[row] [col]) 


Hit, 2) ttt 
#[3, 4] 
#15, 6, 7] 
#18, 9, 16] 
#15 


AM 1 14 
Using slices in nested list: 


print(alist[1][1:]) 
calls oe AUG, Uo Ne Te Za 
#Slices still work 


The final list: 


print(alist) 
AU dy en. lily Heke Za HS Gy Zl, ISSO, UGH, US, 12, We. aes 


Section 20.13: Initializing a List to a Fixed Number of Elements 


For immutable elements (e.g. None, string literals etc.): 


my_list = [None] * 10 
my_list = ['test'] * 10 


For mutable elements, the same construct will result in all elements of the list referring to the same object, for 
example, for a set: 


>>> my_list=[{1}] * 18 
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>>> print(my_list) 


ner ptt git dae thee hoe Mt) eed Ey eet etal 
>>> my_list[@].add(2) 
>>> print(my_list) 


PRC 74 ele Aer ean eter sire” om 78 oe een tll stele Millen 2 ece aul 29a 
Instead, to initialize the list with a fixed number of different mutable objects, use: 


my_list=[{1} for _ in range(1@)] 
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Chapter 21: List comprehensions 


List comprehensions in Python are concise, syntactic constructs. They can be utilized to generate lists from other 
lists by applying functions to each element in the list. The following section explains and demonstrates the use of 
these expressions. 


Section 21.1: List Comprehensions 


A list comprehension creates a new list by applying an expression to each element of an iterable. The most basic 
form is: 


[ <expression> for <element> in <iterable> ] 
There's also an optional 'if' condition: 
[ <expression> for <element> in <iterable> if <condition> ] 
Each <element> in the <iterable> is plugged in to the <expression> if the (optional) <condition> evaluates to true 


. All results are returned at once in the new list. Generator expressions are evaluated lazily, but list comprehensions 
evaluate the entire iterator immediately - consuming memory proportional to the iterator's length. 


To create a list of squared integers: 


squares = [x * x for x in (1, 2, 3, 4)] 
# squares: [1, 4, 9, 16] 


The for expression sets x to each value in turn from (1, 2, 3, 4). The result of the expression x * x is appended 
to an internal list. The internal list is assigned to the variable squares when completed. 


Besides a speed increase (as explained here), a list comprehension is roughly equivalent to the following for-loop: 


squares = [] 

for xin, 2. 3.) 4) 
squares.append(x * x) 

# squares: [1, 4, 9, 16] 


The expression applied to each element can be as complex as needed: 


# Get a list of uppercase characters from a string 

[s.upper() for s in “Hello World" ] 

7a a eset Fs |e | ec) a san A () ar ee | Ce) 

# Strip off any commas from the end of strings in a list 

[w.strip(',') for w in ['these,', 'words,,', ‘mostly’, ‘have,commas,']] 

# ['these', ‘words’, ‘mostly’, ‘have, commas’ ] 

# Organize letters in words more reasonably - in an alphabetical order 

sentence = "Beautiful is better than ugly" 

["".join(sorted(word, key = lambda x: x.lower())) for word in sentence.split()] 
# ['aBefiltuu', ‘is’, ‘beertt', ‘ahnt', ‘gluy'] 


else 


else can be used in List comprehension constructs, but be careful regarding the syntax. The if/else clauses should 
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be used before for loop, not after: 


# create a list of characters in apple, replacing non vowels with 'x' 
fe Ex - ‘apple’ aos eas. ae Tee Peepal , ed 


[x for x in ‘apple’ if x in ‘aeiou' else 'x'] 
#SyntaxError: invalid syntax 


# When using if/else together use them before the loop 
[x if x in 'aeiou' else '*' for x in ‘apple'] 
ae erie a er eee | 


Note this uses a different language construct, a conditional expression, which itself is not part of the 
comprehension syntax. Whereas the if after the for...in /s a part of list comprehensions and used to filter 
elements from the source iterable. 


Double Iteration 


Order of double iteration[... for x in ... for y in ...] is either natural or counter-intuitive. The rule of 
thumb is to follow an equivalent for loop: 


def foo(i): 
return i, i+ @.5 


for i in range(3): 
for x in foo(i): 
yield str(x) 


This becomes: 


[str(x) 
for i in range(3) 
for x in foo(i) 


This can be compressed into one line as [str(x) for i in range(3) for x in foo(i)] 


In-place Mutation and Other Side Effects 


Before using list comprehension, understand the difference between functions called for their side effects 
(mutating, or in-place functions) which usually return None, and functions that return an interesting value. 


Many functions (especially pure functions) simply take an object and return some object. An in-place function 
modifies the existing object, which is called a side effect. Other examples include input and output operations such 
as printing. 


list.sort() sorts a list in-place (meaning that it modifies the original list) and returns the value None. Therefore, it 
won't work as expected in a list comprehension: 


[x.sort() for x in [[2, 1], [4, 3], [@, 1]]] 
# [None, None, None] 


Instead, sorted() returns a sorted list rather than sorting in-place: 
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[sorted(x) for x in [[2, 1], [4, 3], [@, 1]]] 
PU Zip len Aho len Ih) 


Using comprehensions for side-effects is possible, such as I/O or in-place functions. Yet a for loop is usually more 
readable. While this works in Python 3: 


[print(x) for x in (1, 2, 3)] 
Instead use: 


for xein (28 3) 
print(x) 


In some situations, side effect functions are suitable for list comprehension. random. randrange() has the side 
effect of changing the state of the random number generator, but it also returns an interesting value. Additionally, 
next() can be called on an iterator. 


The following random value generator is not pure, yet makes sense as the random generator is reset every time the 
expression is evaluated: 


from random import randrange 
[randrange(1, 7) for _ in range(18@)] 
# IZ, 3) 25 ly Uh i 2 4, 3; 5] 


Whitespace in list comprehensions 


More complicated list comprehensions can reach an undesired length, or become less readable. Although less 
common in examples, it is possible to break a list comprehension into multiple lines like so: 


x for x 
in '‘foo' 
if x not in ‘bar' 


Section 21.2: Conditional List Comprehensions 


Given a list comprehension you can append one or more if conditions to filter values. 


[<expression> for <element> in <iterable> if <condition>] 


For each <element> in <iterable>; if <condition> evaluates to True, add <expression> (usually a function of 
<element>) to the returned list. 


For example, this can be used to extract only even numbers from a sequence of integers: 


[x for x in range(10) if x % 2 == 6] 
# Out: [@, 2, 4, 6, 8] 


Live demo 
The above code is equivalent to: 


even_numbers = [] 
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for x in range(1@): 
if x % 2 == 
even_numbers.append(x) 


print(even_numbers) 
ALOU Oye 2un 4 Or Oi) 


Also, a conditional list comprehension of the form [e for x in y if c] (wheree and c are expressions in terms of 
x) is equivalent to list(filter(lambda x: c, map(lambda x: e, y))). 


Despite providing the same result, pay attention to the fact that the former example is almost 2x faster than the 
latter one. For those who are curious, this is a nice explanation of the reason why. 


Note that this is quite different from the ... if ... else ... conditional expression (sometimes known as a 
ternary expression) that you can use for the <expression> part of the list comprehension. Consider the following 
example: 


[x if x % 2 == @ else None for x in range(1@)] 
# Out: [@, None, 2, None, 4, None, 6, None, 8, None] 


Live demo 


Here the conditional expression isn't a filter, but rather an operator determining the value to be used for the list 
items: 


<value-if-condition-is-true> if <condition> else <value-if-condition-is-false> 
This becomes more obvious if you combine it with other operators: 


[2 * (x if x % 2 == @ else -1) + 1 for x in range(1@)] 
ASOUte ie = 1000, dh Oe = Teo ede 7, =) 


Live demo 


If you are using Python 2.7, xrange may be better than range for several reasons as described in the xrange 
documentation. 


[2 * (x if x % 2 == @ else -1) + 1 for x in xrange(1@)] 
i2 One (ie Sil, Sy ath oe aie iste aie lie =p) 


The above code is equivalent to: 


numbers = [] 
for x in range(1@): 
if x % 2 == 
temp = x 
else: 
temp = -1 
numbers.append(2 * temp + 1) 
print(numbers) 
# Out: [1, -1, 5, -1, 9, -1, 13, -1, 17, -1] 


One can combine ternary expressions and if conditions. The ternary operator works on the filtered result: 


[x if x > 2 else '*' for x in range(10) if x % 2 == @] 
fe (OWNER fl ee 4 Per, Cie Toe itch) 
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The same couldn't have been achieved just by ternary operator only: 


[x if (x > 2 and x % 2 == 0) else ‘*' for x in range(10)] 
Fe OUits 5 | ae tee ot ena gene Os A 


, , fi 


See also: Filters, which often provide a sufficient alternative to conditional list comprehensions. 


Section 21.3: Avoid repetitive and expensive operations using 
conditional clause 


Consider the below list comprehension: 


>>> def f(x): 
import time 
time.sleep(.1) # Simulate expensive function 
return x**2 


>>> [f(x) for x in range(1000) if f(x) > 10] 
(16, 25, S62] 


This results in two calls to f(x) for 1,000 values of x: one call for generating the value and the other for checking the 
if condition. If f(x) is a particularly expensive operation, this can have significant performance implications. 
Worse, if calling f() has side effects, it can have surprising results. 


Instead, you should evaluate the expensive operation only once for each value of x by generating an intermediate 
iterable (generator expression) as follows: 


>>> [v for v in (f(x) for x in range(1000)) if v > 10] 
1625) S6ray |] 


Or, using the builtin map equivalent: 


>>> [v for v in map(f, range(1000)) if v > 10] 
LG, Sy SIS See 


Another way that could result in a more readable code is to put the partial result (v in the previous example) in an 
iterable (such as a list or a tuple) and then iterate over it. Since v will be the only element in the iterable, the result is 
that we now have a reference to the output of our slow function computed only once: 


>>> [v for x in range(100@) for v in [f(x)] if v > 16] 
[1G 25.0 864] 


However, in practice, the logic of code can be more complicated and it's important to keep it readable. In general, a 
separate generator function is recommended over a complex one-liner: 


>>> def process_prime_numbers(iterable) : 
for x in iterable: 
if is_prime(x): 
yield f(x) 


>>> [x for x in process_prime_numbers(range(1000@)) if x > 10] 
Pili’ Ali aoe APs. eacceel| 


Another way to prevent computing f(x) multiple times is to use the @functools.1ru_cache()(Python 3.2+) 
decorator on f (x). This way since the output of f for the input x has already been computed once, the second 
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function invocation of the original list comprehension will be as fast as a dictionary lookup. This approach uses 
memoization to improve efficiency, which is comparable to using generator expressions. 


Say you have to flatten a list 
We hy, Be Si, ee Se alle (aly Wey Sil 
Some of the methods could be: 


reduce(lambda x, y: x+y, 1) 
sum(1, []) 


list(itertools.chain(+1) ) 
However list comprehension would provide the best time complexity. 
[item for sublist in 1 for item in sublist] 


The shortcuts based on + (including the implied use in sum) are, of necessity, O(L42) when there are L sublists -- as 
the intermediate result list keeps getting longer, at each step a new intermediate result list object gets allocated, 
and all the items in the previous intermediate result must be copied over (as well as a few new ones added at the 
end). So (for simplicity and without actual loss of generality) say you have L sublists of | items each: the first | items 
are copied back and forth L-1 times, the second | items L-2 times, and so on; total number of copies is | times the 
sum of x for x from 1 to L excluded, i.e., | * (L**2)/2. 


The list comprehension just generates one list, once, and copies each item over (from its original place of residence 
to the result list) also exactly once. 


Section 21.4: Dictionary Comprehensions 


A dictionary comprehension is similar to a list comprehension except that it produces a dictionary object instead of 
a list. 


A basic example: 


Python 2.x Version = 2.7 
{x xi * x for x in (1, 2, 3, 4)} 


Ce URES ALS Ts (Ae ele aE Oe ke TGs 


which is just another way of writing: 


GOBkorn((GG oe ce SO) Sil os Hil, an Sie 0) 
U2 Ome ee a, eel epee ey Zi rie 


As with a list comprehension, we can use a conditional statement inside the dict comprehension to produce only 
the dict elements meeting some criterion. 


Python 2.x Version = 2.7 
{name: len(name) for name in ('Stack', 'Overflow', 'Exchange') if len(name) > 6} 
# Out: {'Exchange': 8, ‘Overflow’: 8} 


Or, rewritten using a generator expression. 
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dict((name, len(name)) for name in ('Stack', 'Overflow', 'Exchange') if len(name) > 6) 
# Out: {'Exchange': 8, ‘Overflow’: 8} 


Starting with a dictionary and using dictionary comprehension as a key-value pair filter 


Python 2.x Version = 2.7 

initial_dict = {'x': 1, ‘y': 2} 

{key: value for key, value in initial_dict.items() if key == 'x'} 
ue nee a oe ly 


Switching key and value of dictionary (invert dictionary) 

If you have a dict containing simple hashable values (duplicate values may have unexpected results): 

MyxdtCt == {he a2 a Die Sc emeCey 

and you wanted to swap the keys and values you can take several approaches depending on your coding style: 


® swapped = {v: k for k, v in my_dict.items()} 

¢ swapped = dict((v, k) for k, v in my_dict.iteritems() ) 
® swapped = dict(zip(my_dict.values(), my_dict)) 

© swapped = dict(zip(my_dict.values(), my_dict.keys())) 
© swapped = dict(map(reversed, my_dict.items())) 


print (swapped) 
u2 (One Ele tle Ios Be om Sy 


Python 2.x Version = 2.3 


If your dictionary is large, consider importing itertoo/s and utilize izip or imap. 


Merging Dictionaries 
Combine dictionaries and optionally override old values with a nested dictionary comprehension. 


1} 


ohWonm) yep i me 
Ne a ae es Pe) rea Pa 


{ 
dict2 = {' 


{k: v for d in [dict1, dict2] for k, v in d.items()} 
HO UE iee an Wine atl) eX Ge SZ s/t ee Ze 


However, dictionary unpacking (PEP 448) may be a preferred. 


Python 3.x Version = 3.5 


{*xxdict1, **dict2} 
Hea Uitte (ha Wins EG e/a eR Zale EZ 


Note: dictionary comprehensions were added in Python 3.0 and backported to 2.7+, unlike list comprehensions, 
which were added in 2.0. Versions < 2.7 can use generator expressions and the dict() builtin to simulate the 
behavior of dictionary comprehensions. 


Section 21.5: List Comprehensions with Nested Loops 


List Comprehensions can use nested for loops. You can code any number of nested for loops within a list 
comprehension, and each for loop may have an optional associated if test. When doing so, the order of the for 
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constructs is the same order as when writing a series of nested for statements. The general structure of list 
comprehensions looks like this: 


[ expression for target1 in iterable1 [if condition1] 
for target2 in iterable2 [if condition2]... 
for targetN in iterableN [if conditionN] ] 


For example, the following code flattening a list of lists using multiple for statements: 


datal="([t 2iy iss 41) [5.6] 
output = [] 
for each_list in data: 
for element in each_list: 
output.append(element) 
print(output) 
ca Ones |i, 2, eh 2s Sy, il 


can be equivalently written as a list comprehension with multiple for constructs: 


data = [1 21, [3) 41, (5, 6ll 

output = [element for each_list in data for element in each_list] 
print(output) 

2 Ones Ile 2 She Ze Sy Coll 


Live Demo 


In both the expanded form and the list comprehension, the outer loop (first for statement) comes first. 


In addition to being more compact, the nested comprehension is also significantly faster. 


In [1]: data = [[1,2],[3,4],[5,6]] 
In [2]: def f(): 

= output=[ ] 

for each_list in data: 
for element in each_list: 
output. append(element ) 

Rede return output 
In [3]: timeit f() 
1000008 loops, best of 3: 1.37 us per loop 
In [4]: timeit [inner for outer in data for inner in outer] 
1080008 loops, best of 3: 632 ns per loop 


The overhead for the function call above is about 740ns. 


Inline ifs are nested similarly, and may occur in any position after the first for: 


datac= fila [2S 45. Sill] 

output = [element for each_list in data 
if len(each_list) == 2 
for element in each_list 
if element != 5] 

print(output) 

ce ORE Pon Sin 4 


Live Demo 


For the sake of readability, however, you should consider using traditional for-loops. This is especially true when 


nesting is more than 2 levels deep, and/or the logic of the comprehension is too complex. multiple nested loop list 
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comprehension could be error prone or it gives unexpected result. 


Section 21.6: Generator Expressions 


Generator expressions are very similar to list comprehensions. The main difference is that it does not create a full 
set of results at once; it creates a generator object which can then be iterated over. 


For instance, see the difference in the following code: 


# list comprehension 

[x**2 for x in range(1@)] 

# Output: [@, 1, 4, 9, 16, 25, 36, 49, 64, 81] 
Python 2.x Version = 2.4 


# generator comprehension 
(x**2 for x in xrange(1@)) 
# Output: <generator object <genexpr> at @x11b4b7c8@> 


These are two very different objects: 


e the list comprehension returns a list object whereas the generator comprehension returns a generator. 


e generator objects cannot be indexed and makes use of the next function to get items in order. 


Note: We use xrange since it too creates a generator object. If we would use range, a list would be created. Also, 
xrange exists only in later version of python 2. In python 3, range just returns a generator. For more information, 
see the Differences between range and xrange functions example. 


Python 2.x Version = 2.4 


g = (x**2 for x in xrange(1@)) 
print(g[@]) 


Traceback (most recent call last): 
File "<stdin>", line 1, in <module> 
TypeError: ‘generator’ object has no attribute 


__getitem__' 


g.next() #84 
.next() #7 
g.next() #4 


a 


g.next() # 81 


g.next() # Throws StopIteration Exception 


Traceback (most recent call last): 
File "<stdin>", line 1, in <module> 
StopIteration 


Python 3.x Version = 3.0 


NOTE: The function g.next() should be substituted by next(g) and xrange with range since 
Iterator .next() and xrange() do not exist in Python 3. 


Although both of these can be iterated in a similar way: 
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for 


i in [x**2 for x in range(10)]: 
print(i) 


Python 2.x Version = 2.4 


for 


OuEe 


hk - © 


81 


i in (x**2 for x in xrange(1@)): 
print(i) 


Use cases 


Generator expressions are lazily evaluated, which means that they generate and return each value only when the 
generator is iterated. This is often useful when iterating through large datasets, avoiding the need to create a 
duplicate of the dataset in memory: 


for 


square in (x**2 for x in range(1000000)): 
#do something 


Another common use case is to avoid iterating over an entire iterable if doing so is not necessary. In this example, 
an item is retrieved from a remote API with each iteration of get_objects(). Thousands of objects may exist, must 
be retrieved one-by-one, and we only need to know if an object matching a pattern exists. By using a generator 
expression, when we encounter an object matching the pattern. 


def 


def 


def 


get_objects(): 
"""Gets objects from an API one by one 
while True: 

yield get_next_item() 


object_matches_pattern(obj) : 
# perform potentially complex calculation 
return matches_pattern 


right_item_exists(): 
items = (object_matched_pattern(each) for each in get_objects()) 
for item in items: 

if item.is_the_right_one: 


return True 
return False 
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Section 21.7: Set Comprehensions 


Set comprehension is similar to list and dictionary comprehension, but it produces a set, which is an unordered 
collection of unique elements. 


Python 2.x Version = 2.7 


# A set containing every value in range(5): 
{x for x in range(5)} 
ce (Olnes 3S), th 2a ely 


# A set of even numbers between 7 and 1@: 
{x for x in range(1, 11) if x % 2 == Q} 
# Outs 42-4, 16, 8. 110}} 


# Unique alphabetic characters in a string of text: 
text = "When in the Course of human events it becomes necessary for one people..." 
{ch.lower() for ch in text if ch.isalpha()} 

gf (OURS: SNR MEE eR Elo, SG PER USE ye it OA Ute 
# "i fe gp SE ie Ge Te I, ONE PP I), 


, 


Live Demo 


Keep in mind that sets are unordered. This means that the order of the results in the set may differ from the one 
presented in the above examples. 


Note: Set comprehension is available since python 2.7+, unlike list comprehensions, which were added in 2.0. In 
Python 2.2 to Python 2.6, the set() function can be used with a generator expression to produce the same result: 


Python 2.x Version = 2.2 


set(x for x in range(5)) 
HOULEE (Glas 2a Shay 


Section 21.8: Refactoring filter and map to list 
comprehensions 


The filter or map functions should often be replaced by list comprehensions. Guido Van Rossum describes this 
well in an open letter in 2005: 


filter(P, S) is almost always written clearer as [x for x in S if P(x)], and this has the huge 
advantage that the most common usages involve predicates that are comparisons, e.g. x==42, and 
defining a lambda for that just requires much more effort for the reader (plus the lambda is slower than 
the list comprehension). Even more so for map(F, S) which becomes [F(x) for x in S$]. Of course, in 
many cases you'd be able to use generator expressions instead. 


The following lines of code are considered "not pythonic" and will raise errors in many python linters. 


filter(lambda x: x % 2 == @, range(10)) # even numbers < 16 
map(lambda x: 2*x, range(10)) # multiply each number by two 
reduce(lambda x,y: xty, range(10)) # sum of all elements in list 


Taking what we have learned from the previous quote, we can break down these filter and map expressions into 


their equivalent list comprehensions; also removing the lambda functions from each - making the code more 
readable in the process. 
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# Filter: 
# P(x) = x %2 == @ 
# S = range(18@) 


[x for x in range(1@) if x % 2 == @] 
# Map 
HFK) = 2EX 


# S = range(18@) 
[2*x for x in range(10)] 


Readability becomes even more apparent when dealing with chaining functions. Where due to readability, the 
results of one map or filter function should be passed as a result to the next; with simple cases, these can be 
replaced with a single list comprehension. Further, we can easily tell from the list comprehension what the outcome 
of our process is, where there is more cognitive load when reasoning about the chained Map & Filter process. 


# Map & Filter 
filtered = filter(lambda x: x % 2 == @, range(1@)) 
results = map(lambda x: 2%*x, filtered) 


# List comprehension 
results = [2*x for x in range(1@) if x % 2 == @] 


Refactoring - Quick Reference 


e Map 
map(F, S) == [F(x) for x in S$] 
e Filter 
filter(P, S) == [x for x in S if P(x)] 


where F and P are functions which respectively transform input values and return a bool 


Section 21.9: Comprehensions involving tuples 


The for clause of a list comprehension can specify more than one variable: 


[x + y for x, y in [(1, 2), (3, 4), (5, 6)]] 
ca (Olunee Sh 2,9 bal 


[x + y for x, y in zip([1, 3, 5], [2, 4, 6])] 
# Outs 3, 7, 1 


This is just like regular for loops: 


for x, y in [(1,2), (3,4). (,6)1: 
print(xty) 

# 3 

# 7 

# 11 


Note however, if the expression that begins the comprehension is a tuple then it must be parenthesized: 
oye hero. yo dm) 2 yo (a4) te, oil 
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# SyntaxError: invalid syntax 


Oey) for-x, yan [(1, 2), (3, 4); (Ss, 6))] 
2 (nee ip, yn (hy CeO), (eS, il 


Section 21.10: Counting Occurrences Using Comprehension 


When we want to count the number of items in an iterable, that meet some condition, we can use comprehension 
to produce an idiomatic syntax: 


# Count the numbers in ‘range(1@@@)° that are even and contain the digit ‘9°: 
print (sum( 
1 for x in range(1000) 
if x % 2 == @ and 
'9' in str(x) 
)) 
# Out: 95 


The basic concept can be summarized as: 


1. Iterate over the elements in range(10@@). 

2. Concatenate all the needed if conditions. 

3. Use 1 as expression to return a 1 for each item that meets the conditions. 
4, Sum up all the 1s to determine number of items that meet the conditions. 


Note: Here we are not collecting the 1s in a list (note the absence of square brackets), but we are passing the ones 


directly to the sum function that is summing them up. This is called a generator expression, which is similar to a 
Comprehension. 


Section 21.11: Changing Types in a List 


Quantitative data is often read in as strings that must be converted to numeric types before processing. The types 
of all list items can be converted with either a List Comprehension or the map() function. 


# Convert a list of strings to integers. 
TEMS e—si nile 2S ee Aca] 

[int(item) for item in items] 

A Outs i, 2) <3, 4] 


# Convert a list of strings to float. 
aiteMs = alot no, Al 

map(float, items) 

A MOtnesiieGy Ast, Sia cach) 


Section 21.12: Nested List Comprehensions 


Nested list comprehensions, unlike list comprehensions with nested loops, are List comprehensions within a list 
comprehension. The initial expression can be any arbitrary expression, including another list comprehension. 


#List Comprehension with nested loop 
[x + y for x in [1, 2, 3] for y in [3, 4, 5]] 
PAQUES 12h ys (py Wy Oy in (8, Te Ch) 


#Nested List Comprehension 


[[x + y for x in [1, 2, 3]] for y in [3, 4, 5]] 
comes IleZk By oll, [ey py wlln HG, 76, GHill 
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The Nested example is equivalent to 


1= [] 
for y in [3, 4, 5): 
temp = [] 
for <an: [i 2) 3): 
temp.append(x + y) 
1.append( temp) 


One example where a nested comprehension can be used it to transpose a matrix. 


matrix = [[1,2,3], 
[41576 


[7,8,9]] 


[[row[i] for row in matrix] for i in range(len(matrix)) ] 
ee UE Can Aly Ua ey ee Mi Te EU 


Like nested for loops, there is no limit to how deep comprehensions can be nested. 


[{{i + j + k for k in 'cd'] for j in ‘'ab'] for i in ‘'12'] 


HOt wie aCumealadallne| be a lbduli) wali 2ac. aZad slau 2DC\ ee 2b all 


Section 21.13: Iterate two or more list simultaneously within 
list comprehension 


For iterating more than two lists simultaneously within /ist comprehension, one may use zip() as: 


Seer dates (ily 2, ke 2 

SSS ikbigwy 2 a, tb, "ee, “ol 

So> slaistaga=—\(O 2 2OL Ow | 

# Two lists 

>>> [(i, j) for i, j in zip(list_1, list_2)] 
te ae 2 A 8 eo) a did 


# Three lists 
o>) (ii gin k)s fOr L, 9), Kain zipdistel Wiste25) dais te3)i] 
Tes Re OG (25, De ete, a Baas ds 2.) 


# soon... 
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Chapter 22: List slicing (selecting parts of 
lists) 


Section 22.1: Using the third "step” argument 
lst = [eae “ilo). KCum “ols ‘e', ean els linia 


1st[::2] 

#LOULDUET a an 4C4 muse ee Gill 
1st[::3] 

# Outputs lai die sg) 


Section 22.2: Selecting a sublist from a list 
ee Shey oy ee ch ee 


lst [2:4] 
2 OMG iee Ie. Vol tl 


Ist[2:] 

ie Otnies 26%, tl, “2 )] 
Ist[:4] 

ce Olnoyines {ey Sly oe Meal || 


Section 22.3: Reversing a list with slicing 
a= [lp 2 Sy 2 Sl 


# steps through the list backwards (step=-1) 
b = a[::-1] 


# built-in list method to reverse ‘a’ 
a.reverse() 


if a=b: 
print(True) 


print(b) 
# Output: 


# True 
#5, 4553, 25 Tf 


Section 22.4: Shifting a list using slicing 


def shift_list(array, s): 
"""Shifts the elements of a list to the left or right. 


Args: 

array - the list to shift 

s - the amount to shift the list ('+': right-shift, '-': left-shift) 
Returns: 


shifted_array - the shifted list 
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# calculate actual shift amount (e.g., 11 --> 1 if length of the array is 5) 
s %= len(array) 


# reverse the shift direction to be more intuitive 
S *= -1 


# shift array with list slicing 
shifted_array = array[s:] + array[:s] 


return shifted_array 
my_array = [1, 2, 3, 4, 5] 
# negative numbers 
shift_list(my_array, -7) 
==> [6,4 5, 1, 2) 
# no shift on numbers equal to the size of the array 
shift_list(my_array, 5) 
s= (1, 2, 3, 4,5) 
# works on positive numbers 


shift_list(my_array, 3) 
eee [8,°4; 5, 1,2] 
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Chapter 23: groupby() 


Parameter Details 
iterable Any python iterable 


key Function(criteria) on which to group the iterable 


In Python, the itertools.groupby() method allows developers to group values of an iterable class based ona 
specified property into another iterable set of values. 


Section 23.1: Example 4 
In this example we see what happens when we use different types of iterable. 


things = [("animal", “bear"), ("animal", "“duck"), ("plant", "cactus"), ("vehicle", "harley"), \ 
("vehicle", "speed boat"), ("vehicle", “school bus")] 
dic = {} 
f = lambda x: x[@] 
for key, group in groupby(sorted(things, key=f), f): 
dic[key] = list(group) 


dic 

Results in 

{'animal': [('animal', ‘bear'), (‘animal', '‘duck')] 
‘plant': [('plant', ‘cactus')] 
‘vehicle’: [('vehicle', ‘harley'), 


('vehicle', ‘speed boat’), 
('vehicle', ‘school bus')]} 


This example below is essentially the same as the one above it. The only difference is that | have changed all the 
tuples to lists. 


things = [["animal", "bear"], ["animal", "duck"], ["vehicle", “harley"], ["“plant", "cactus"], \ 
["vehicle", "speed boat"], ["vehicle", "school bus"]] 
dic = {} 
f = lambda x: x[@] 
for key, group in groupby(sorted(things, key=f), f): 
dic[key] = list(group) 


dic 
Results 
{'animal': [['animal', 'bear'], ['animal', ‘duck']] 
Jplants[[Uplanty.. “cactusia||| 
‘vehicle’: [['vehicle', ‘harley'], 
['vehicle', ‘speed boat'], 
['vehicle', ‘school bus']]} 


Section 23.2: Example 2 


This example illustrates how the default key is chosen if we do not specify any 


c = groupby(['goat', ‘dog', ‘cow’, 1, 1, 2, 3, 11, 18, (‘persons', 'man', ‘woman’ )]) 
dic = {} 
for k, v inc: 


Goalkicker.com - Python® Notes for Professionals 146 


dic[k] = list(v) 
dic 


Results in 


see ile ly 

2421, 

a: [3], 

(‘persons', ‘man', ‘woman'): [('persons', ‘man’, '‘woman')] 
‘cow': ['cow'], 

idog: = s[dog: |r 

10: [10], 

flelesen Dollie 

tgoat+ = [| -goat ]/} 


Notice here that the tuple as a whole counts as one key in this list 


Section 23.3: Example 3 


Notice in this example that mulato and camel don't show up in our result. Only the last element with the specified 
key shows up. The last result for c actually wipes out two previous results. But watch the new version where | have 
the data sorted first on same key. 


list_things = ['goat', ‘dog', ‘donkey’, ‘mulato', ‘cow', ‘cat', ('‘persons', 'man', ‘woman'), \ 
‘wombat’, ‘mongoose’, ‘malloo', ‘camel’ ] 
c = groupby(list_things, key=lambda x: x[@]) 
dic = {} 
for k, v inc: 
dic[k] = list(v) 

dic 
Results in 
{cw ailimeamedal|: 

'd': ['dog', ‘donkey'], 

“gl goat]; 

‘m': ['mongoose', ‘malloo'], 

‘persons': [('persons', ‘man', ‘woman' )] 

‘w': ['wombat' ]} 
Sorted Version 
list_things = ['goat', ‘'dog', ‘donkey’, ‘mulato', ‘cow', ‘cat', ('persons', 'man', ‘woman'), \ 

‘wombat’, ‘mongoose’, ‘malloo', ‘camel’ ] 
sorted_list = sorted(list_things, key = lambda x: x[@]) 
print(sorted_list) 
print() 
c = groupby(sorted_list, key=lambda x: x[@]) 
dic = {} 
for k, v inc: 
dic[k] = list(v) 

dic 
Results in 
['cow', ‘cat', ‘camel’, ‘dog', ‘donkey', ‘goat', ‘mulato', ‘mongoose’, ‘malloo', ('persons', ‘man', 


‘woman'), ‘wombat’ ] 
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{eT Cesc ltCOWL a Caitu mn camel alle 
tdi \[(sdogy, “donkey ® |j, 


ig. logeat |, 
‘m': ['mulato', 'mongoose', ‘malloo'], 
‘persons’: [('persons', 'man', ‘woman')] 


‘w': ['wombat' ]} 
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Chapter 24: Linked lists 


A linked list is a collection of nodes, each made up of a reference and a value. Nodes are strung together into a 
sequence using their references. Linked lists can be used to implement more complex data structures like lists, 
stacks, queues, and associative arrays. 


Section 24.1: Single linked list example 


This example implements a linked list with many of the same methods as that of the built-in list object. 


class Node: 
def __init__(self, val): 
self.data = val 
self.next = None 


def getData(self): 
return self .data 


def getNext(self): 
return self .next 


def setData(self, val): 
self.data = val 


def setNext(self, val): 
self.next = val 


class LinkedList: 
def __init__(self): 
self.head = None 


def isEmpty(self): 
"""Check if the list is empty 
return self .head is None 


def add(self, item): 
"""Add the item to the list""" 
new_node = Node(item) 
new_node. setNext(self .head) 
self.head = new_node 


def size(self): 
"""Return the length/size of the list""" 
count = @ 
current = self.head 
while current is not None: 
count += 1 
current = current.getNext() 
return count 


def search(self, item): 
"""Search for item in list. If found, return True. If not found, return False 
current = self.head 
found = False 
while current is not None and not found: 
if current.getData() is item: 
found = True 
else: 
current = current.getNext() 
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return found 


def remove(self, item): 
"""Remove item from list. If item is not found in list, raise ValueError 
current = self.head 
previous = None 
found = False 
while current is not None and not found: 
if current.getData() is item: 
found = True 
else: 
previous = current 
current = current.getNext() 
if found: 
if previous is None: 
self.head = current.getNext() 
else: 
previous.setNext(current.getNext()) 


else: 
raise ValueError 
print 'Value not found. ' 


def insert(self, position, item): 
Insert item at position specified. If position specified is 
out of bounds, raise IndexError 
if position > self.size() - 1: 
raise IndexError 
print "Index out of bounds." 
current = self.head 
previous = None 
pos = @ 
if position is @: 
self .add(item) 
else: 
new_node = Node(item) 
while pos < position: 
pos += 1 
previous = current 
current = current.getNext() 
previous. setNext(new_node) 
new_node.setNext(current) 


def index(self, item): 
Return the index where item is found. 
If item is not found, return None. 
current = self.head 
pos = @ 
found = False 
while current is not None and not found: 
if current.getData() is item: 
found = True 
else: 
current = current.getNext() 
pos += 1 
if found: 
pass 
else: 
pos = None 
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return pos 


def pop(self, position = None): 


If no argument is provided, return and remove the item at the head. 
If position is provided, return and remove the item at that position. 
If index is out of bounds, raise IndexError 


if position > self.size(): 
print 'Index out of bounds’ 
raise IndexError 


current = self.head 
if position is None: 
ret = current.getData() 
self.head = current.getNext() 
else: 
pos = @ 
previous = None 
while pos < position: 
previous = current 
current = current.getNext() 
pos += 1 
ret = current.getData() 


previous.setNext(current.getNext()) 


print ret 
return ret 


def append(self, item): 

current = self.head 

previous = None 

pos = @ 

length = self.size() 

while pos < length: 
previous = current 
current = current.getNext() 
pos += 1 

new_node = Node(item) 

if previous is None: 
new_node.setNext(current) 
self.head = new_node 

else: 
previous.setNext(new_node) 


def printList(self): 
Pye lA he alps: dikes ee 
current = self.head 
while current is not None: 
print current.getData() 
current = current.getNext() 


Usage functions much like that of the built-in list. 


11 = LinkedList() 
1ll.add(‘1') 
1l.add(‘H') 
ll.insert(1,‘e') 
11.append('1' ) 
11.append('o' ) 
1l.printList() 
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Append item to the end of the list 


151 
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VIDEO: Complete Python 
Bootcamp: Go trom zero 
to hero in Python 3 


Learn Python like a Professional! Start from the 
basics and go all the way to creating your own 
applications and games! 


COMPLETE PYTHON S 
BOOTCAMP 


v Learn to use Python professionally, learning both Python 2 and Python 3! 

Y¥ Create games with Python, like Tic Tac Toe and Blackjack! 

¥ Learn advanced Python features, like the collections module and how to work with timestamps! 
v Learn to use Object Oriented Programming with classes! 

v Understand complex topics, like decorators. 

v Understand how to use both the Jupyter Notebook and create .py files 

Y Get an understanding of how to create GUls in the Jupyter Notebook system! 

¥Y Build a complete understanding of Python from the ground up! 


Watch Today — 


Chapter 25: Linked List Node 


Section 25.1: Write a simple Linked List Node in python 
A linked list is either: 


e the empty list, represented by None, or 
¢ anode that contains a cargo object and a reference to a linked list. 


#! /usr/bin/env python 


class Node: 
def __init__(self, cargo=None, next=None): 
self.car = cargo 
self.cdr = next 
def __str__(self): 
return str(self.car) 


def display(lst): 
if lst: 
w("%s " % Ist) 
display(lst.cdr) 
else: 
w("nil\n") 
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Chapter 26: Filter 


Parameter Details 
callable that determines the condition or None then use the identity function for filtering (positional- 


function 
only) 


iterable iterable that will be filtered (positional-only) 


Section 26.1: Basic use of filter 
To filter discards elements of a sequence based on some criteria: 
names = ['Fred', 'Wilma', 'Barney'] 

def long_name(name) : 


return len(name) > 5 


Python 2.x Version = 2.0 


filter(long_name, names) 
# Out: ['Barney'] 


[name for name in names if len(name) > 5] # equivalent list comprehension 
# Out: ['Barney'] 


from itertools import ifilter 

ifilter(long_name, names) # as generator (similar to python 3.x filter builtin) 
# Out: <itertools.ifilter at @x4197e10> 

list(ifilter(long_name, names)) # equivalent to filter with lists 

# Out: ['Barney'] 


(name for name in names if len(name) > 5) # equivalent generator expression 
# Out: <generator object <genexpr> at 9x@000000003FD5D38> 


Python 2.x Version = 2.6 


# Besides the options for older python 2.x versions there is a future_builtin function: 
from future_builtins import filter 

filter(long_name, names) # identical to itertools.ifilter 

# Out: <itertools.ifilter at @x3eb@ba8> 


Python 3.x Version = 3.0 


filter(long_name, names) # returns a generator 
# Out: <filter at @x1fc6e44347@> 
list(filter(long_name, names)) # cast to list 

# Out: ['Barney'] 


(name for name in names if len(name) > 5) # equivalent generator expression 
# Out: <generator object <genexpr> at 9@x@00001C6F49BF4C@> 


Section 26.2: Filter without function 


If the function parameter is None, then the identity function will be used: 


list(filter(None, [1, 8, 2, [], '', 'a'])) # discards 6, [] and '' 
ca Onee {ihn Bp EE || 


Python 2.x Version = 2.0.1 
[i for iin [1, 8, 2, [], '', ‘a'] if i] # equivalent list comprehension 


Python 3.x Version = 3.0.0 
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(i for iin [1, 0, 2, [], '', '‘a'] if i) # equivalent generator expression 


Section 26.3: Filter as short-circuit check 


filter (python 3.x) and ifilter (python 2.x) return a generator so they can be very handy when creating a short- 
circuit test like or or and: 


Python 2.x Version = 2.0.1 


# not recommended in real use but keeps the example short: 
from itertools import ifilter as filter 


Python 2.x Version = 2.6.1 


from future_builtins import filter 
To find the first element that is smaller than 100: 


car_shop = [('Toyota', 1000), (‘rectangular tire’, 88), ('Porsche', 500@)] 
def find_something_smaller_than(name_value_tuple) : 

print('Check {0}, {1}$'.format(*name_value_tuple) 

return name_value_tuple[1] < 100 
next(filter(find_something_smaller_than, car_shop) ) 
# Print: Check Toyota, 1000S 
# Check rectangular tire, 80S 
# Out: (‘rectangular tire', 8@) 


The next-function gives the next (in this case first) element of and is therefore the reason why it's short-circuit. 


Section 26.4: Complementary function: filterfalse, ifilterfalse 
There is a complementary function for filter in the itertools-module: 


Python 2.x Version = 2.0.1 


# not recommended in real use but keeps the example valid for python 2.x and python 3.x 
from itertools import ifilterfalse as filterfalse 


Python 3.x Version = 3.0.0 


from itertools import filterfalse 


which works exactly like the generator filter but keeps only the elements that are False: 


# Usage without function (None): 
list(filterfalse(None, [1, 8, 2, [], '', 'a'])) # discards 1, 2, ‘a' 
# Out: (6, [], ~ f 


# Usage with function 
names = ['Fred', ‘Wilma’, ‘Barney’ ] 


def long_name(name) : 
return len(name) > 5 


list(filterfalse(long_name, names) ) 


# Out: ['Fred', ‘Wilma’ ] 


# Short-circuit usage with next: 
car_shop = [('Toyota', 1000), (‘rectangular tire', 88), ('Porsche', 500@)] 
def find_something_smaller_than(name_value_tuple) : 
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print('Check {0}, {1}$'.format(*name_value_tuple) 
return name_value_tuple[1] < 100 
next(filterfalse(find_something_smaller_than, car_shop)) 
# Print: Check Toyota, 1000S 
# Out: ('Toyota', 10@@) 


# Using an equivalent generator: 

car_shop = [('Toyota', 1000), (‘rectangular tire', 88), ('Porsche', 500@)] 
generator = (car for car in car_shop if not car[1] < 100) 

next(generator) 
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Chapter 27: Heapq 


Section 27.1: Largest and smallest items in a collection 


To find the largest items in a collection, heapgq module has a function called nlargest, we pass it two arguments, the 
first one is the number of items that we want to retrieve, the second one is the collection name: 


import heapq 


numbers = [1, 4, 2, 100, 20, 50, 32, 200, 150, 8] 
print(heapq.nlargest(4, numbers)) # [260, 156, 100, 50] 


Similarly, to find the smallest items in a collection, we use nsmallest function: 
print(heapq.nsmallest(4, numbers)) # [1, 2, 4, 8] 


Both nlargest and nsmallest functions take an optional argument (key parameter) for complicated data 
structures. The following example shows the use of age property to retrieve the oldest and the youngest people 
from people dictionary: 


people = [ 
{'firstname': 'John', ‘lastname’: 'Doe', ‘age': 30}, 
{'firstname': 'Jane', ‘lastname’: 'Doe', ‘age’: 25}, 
{'firstname': ‘Janie’, ‘lastname’: 'Doe', ‘age': 10}, 
{'firstname': 'Jane', ‘lastname’: 'Roe', ‘age’: 22}, 
{'firstname': '‘Johnny', ‘lastname’: 'Doe', ‘age’: 12}, 
{'firstname': '‘John', ‘lastname’: 'Roe', ‘age’: 45} 


] 


oldest = heapq.nlargest(2, people, key=lambda s: s['‘age']) 

print(oldest) 

# Output: [{'firstname': ‘John', ‘age': 45, ‘lastname’: 'Roe'}, {'firstname': 'John', ‘age': 38, 
‘lastname’: 'Doe'}] 


youngest = heapq.nsmallest(2, people, key=lambda s: s['‘age']) 
print(youngest) 


# Output: [{'firstname': ‘Janie’, ‘age’: 1@, ‘lastname’: 'Doe'}, {'firstname': '‘Johnny', ‘age': 12, 
‘lastname’: 'Doe'}] 


Section 27.2: Smallest item in a collection 


The most interesting property of a heap is that its smallest element is always the first element: heap[®] 


import heapgq 


numbers = [10, 4, 2, 108, 28, 58, 32, 200, 150, 8] 


heapq.heapify(numbers) 
print(numbers) 
# Output: [2, 4, 10, 100, 8, 50, 32, 200, 150, 20] 


heapq.heappop(numbers) # 2 


print(numbers) 
# Output: [4, 8, 10, 100, 20, 5@, 32, 200, 150] 
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heapq.heappop(numbers) # 4 
print(numbers) 
# Output: [8, 20, 10, 100, 15@, 50, 32, 20@] 


Goalkicker.com - Python® Notes for Professionals 159 


Chapter 28: Tuple 


A tuple is an immutable list of values. Tuples are one of Python's simplest and most common collection types, and 
can be created with the comma operator (value = 1, 2, 3). 


Section 28.1: Tuple 


Syntactically, a tuple is a comma-separated list of values: 


Create an empty tuple with parentheses: 


to® = () 
type(t@) # <type ‘tuple'> 


To create a tuple with a single element, you have to include a final comma: 


wil Ss ely 
type(t1) # <type ‘tuple'> 


Note that a single value in parentheses is not a tuple: 


ta (Gree) 
type(t2) # <type ‘str'> 


To create a singleton tuple it is necessary to have a trailing comma. 


tA) it a) 
type(t2) # <type ‘tuple'> 


Note that for singleton tuples it's recommended (See PEP8 on trailing commas) to use parentheses. Also, no white 
space after the trailing comma (see PEP8 on whitespaces) 


1 (Fel) # PEP8-compliant 
t2 = ‘a’, # this notation is not recommended by PEP8 
t2 = ('a', ) # this notation is not recommended by PEP8 


Another way to create a tuple is the built-in function tuple. 


t = tuple('lupins' ) 


print(t) EP (Cd ig Il a in, SYD) 
t = tuple(range(3)) 
print(t) sane thy 23) 


These examples are based on material from the book Think Python by Allen B. Downey. 
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Section 28.2: Tuples are immutable 


One of the main differences between lists and tuples in Python is that tuples are immutable, that is, one cannot 
add or modify items once the tuple is initialized. For example: 


>>> t= (1, 4, 9) 
>>> t[0] = 2 
Traceback (most recent call last): 
File "<stdin>", line 1, in <module> 
TypeError: ‘tuple’ object does not support item assignment 


Similarly, tuples don't have .append and .extend methods as list does. Using += is possible, but it changes the 
binding of the variable, and not the tuple itself: 


(1, 2) 
t 


+= (3, 4) 


>>> 


(1; 


>>> 


CT, 


INSDEOs IN et eet) och 


Be careful when placing mutable objects, such as lists, inside tuples. This may lead to very confusing outcomes 
when changing them. For example: 


pees decd (lle 4a he (bile 4.0 s1l))) 
(Ciy 25 ee ig 2s 2) 
>>> [3] += [4, 5] 


Will both raise an error and change the contents of the list within the tuple: 


TypeError: ‘tuple’ object does not support item assignment 
SSS ti 


(125-8, 14, 2,38, 47-51) 


You can use the += operator to "append" to a tuple - this works by creating a new tuple with the new element you 
"appended" and assign it to its current variable; the old tuple is not changed, but replaced! 


This avoids converting to and from a list, but this is slow and is a bad practice, especially if you're going to append 
multiple times. 


Section 28.3: Packing and Unpacking Tuples 


Tuples in Python are values separated by commas. Enclosing parentheses for inputting tuples are optional, so the 
two assignments 


a= (aeeaS # ais the tuple (1, 2, 3) 


a=" (1, 2) 3) # alas the tuple (i, 2, 3) 
are equivalent. The assignmenta = 1, 2, 3is also called packing because it packs values together in a tuple. 


Note that a one-value tuple is also a tuple. To tell Python that a variable is a tuple and not a single value you can use 
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a trailing comma 


a= 1 #a iis the value 17 
a= 1, # ais the tuple (7, ) 


A comma is needed also if you use parentheses 


(1,) # ais the tuple (1,) 
a= (1) # ais the value 7 and not a tuple 


fe) 
Ul 


To unpack values from a tuple and do multiple assignments use 


# unpacking AKA multiple assignment 
x, y, Z = (1, 2 3) 


eR 
NN X 
I 
t 
oO hM-a 


The symbol _ can be used as a disposable variable name if one only needs some elements of a tuple, acting asa 
placeholder: 


Single element tuples: 


1, #x is the value 7 
1, #x is the tuple (1,) 


xX, 
x 


In Python 3 a target variable with a * prefix can be used as a catch-all variable (see Unpacking Iterables ): 


Python 3.x Version = 3.0 
first, *more, last = (1, 2, 3, 4, 5) 


2 ee Syl 
# more == [2, 3, 4] 
Holast==— 5, 


Section 28.4: Built-in Tuple Functions 


Tuples support the following build-in functions 
Comparison 


If elements are of the same type, python performs the comparison and returns the result. If elements are different 
types, it checks whether they are numbers. 


e If numbers, perform comparison. 
e If either element is a number, then the other element is returned. 
e Otherwise, types are sorted alphabetically . 


If we reached the end of one of the lists, the longer list is "larger." If both list are same it returns 0. 


tuple1 = 


Ga “bine Cae Cia ‘e') 
tuple2 = (‘1 


Doe 28) 
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tuple3 = (‘a', pie ow vd Tee) 


cmp(tuple1, tuple2) 
Out: 1 


cmp(tuple2, tuple) 
Out: -1 


cmp(tuple1, tuple3) 
Out: @ 


Tuple Length 
The function len returns the total length of the tuple 


len(tuple1) 
Out: 5 


Max of a tuple 
The function max returns item from the tuple with the max value 


max(tuple1) 


Out: ‘e 
max(tuple2) 
Out: ‘3° 


Min of a tuple 
The function min returns the item from the tuple with the min value 


min(tuple1) 


Out: ‘a’ 
min(tuple2) 
Out: ‘1° 


Convert a list into tuple 
The built-in function tuple converts a list into a tuple. 
eiste =" [235i] 


tuple(list) 
Our’ (1, 2, 3,-4.5) 


Tuple concatenation 


Use + to concatenate two tuples 


tuple1 + tuple2 
Out: (‘a', Pbine Me nals ‘e', as hee PERS ois) 


Section 28.5: Tuple Are Element-wise Hashable and Equatable 


hash( (1, 2) ) # ok 
hash( ([], {"hello"}) # not ok, since lists and sets are not hashabe 


Thus a tuple can be put inside a set or as a key in a dict only if each of its elements can. 
{onl 2) ete Ok. 
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{ ([], {"hello"}) ) # not ok 


Section 28.6: Indexing Tuples 


Xe (GI 26 3!) 


x[@] #17 
rhill 22 2 
x[2] #3 


x[3] # IndexError: tuple index out of range 


Indexing with negative numbers will start from the last element as -1: 
x[-1] #3 
x[-2] # 2 


x[-3] #1 
x[-4] # IndexError: tuple index out of range 


Indexing a range of elements 


print(x[:-1])  # (17, 2) 
print(x[-1:]) # (3,) 
print(x[1:3])  # (2, 3) 


Section 28.7: Reversing Elements 


Reverse elements within a tuple 


colors = "red", "green", "blue" 
rev = colors[::-1] 

# rev: ("blue", "green", "red") 
colors = rev 

# colors: ("blue", "green", "red") 


Or using reversed (reversed gives an iterable which is converted to a tuple): 


rev = tuple(reversed(colors)) 


# rev: ("blue", "green", "red") 
colors = rev 
# colors: ("blue", "green", "red") 
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Chapter 29: Basic Input and Output 


Section 29.1: Using the print function 
Python 3.x Version = 3.0 
In Python 3, print functionality is in the form of a function: 


print("This string will be displayed in the output") 
# This string will be displayed in the output 


print("You can print \n escape characters too.") 
# You can print escape characters too. 


Python 2.x Version = 2.3 
In Python 2, print was originally a statement, as shown below. 


print "This string will be displayed in the output" 
# This string will be displayed in the output 


print "You can print \n escape characters too." 
# You can print escape characters too. 


Note: using from __future__ import print_function in Python 2 will allow users to use the print() function the 
same as Python 3 code. This is only available in Python 2.6 and above. 


Section 29.2: Input from a File 


Input can also be read from files. Files can be opened using the built-in function open. Using awith <command> as 
<name> syntax (called a 'Context Manager') makes using open and getting a handle for the file super easy: 


with open('somefile.txt', 'r') as fileobj: 
# write code here using fileobj 


This ensures that when code execution leaves the block the file is automatically closed. 


Files can be opened in different modes. In the above example the file is opened as read-only. To open an existing 
file for reading only use r. If you want to read that file as bytes use rb. To append data to an existing file use a. Use 
w to create a file or overwrite any existing files of the same name. You can use r+ to open a file for both reading and 
writing. The first argument of open() is the filename, the second is the mode. If mode is left blank, it will default to 
r. 


# let's create an example file: 
with open('shoppinglist.txt', 'w') as fileobj: 
fileobj .write('tomato\npasta\ngarlic' ) 


with open('shoppinglist.txt', 'r') as fileobj: 
# this method makes a list where each line 


# of the file is an element in the list 
lines = fileobj.readlines() 


print(lines) 
# ['tomato\n', 'pasta\n', ‘garlic’ ] 


with open('shoppinglist.txt', 'r') as fileobj: 
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# here we read the whole content into one string: 

content = fileobj.read() 

# get a list of lines, just like int the previous example: 
lines = content.split('\n') 


print(lines) 
# ['tomato', ‘pasta’, ‘garlic’ ] 


If the size of the file is tiny, it is safe to read the whole file contents into memory. If the file is very large it is often 
better to read line-by-line or by chunks, and process the input in the same loop. To do that: 


with open('shoppinglist.txt', 'r') as fileobj: 
# this method reads line by line: 
lines = [] 
for line in fileobj: 
lines .append(line.strip() ) 


When reading files, be aware of the operating system-specific line-break characters. Although for line in 
fileobj automatically strips them off, it is always safe to call strip() on the lines read, as it is shown above. 


Opened files (fileobj in the above examples) always point to a specific location in the file. When they are first 
opened the file handle points to the very beginning of the file, which is the position 8. The file handle can display its 
current position with tell: 


fileobj = open('shoppinglist.txt', ‘r') 
pos = fileobj.tell() 
print('We are at %u.' % pos) # We are at @. 


Upon reading all the content, the file handler's position will be pointed at the end of the file: 


content = fileobj.read() 

end = fileobj.tell() 

print('This file was %u characters long.' % end) 
# This file was 22 characters long. 
fileobj.close() 


The file handler position can be set to whatever is needed: 


fileobj = open('shoppinglist.txt', ‘r') 
fileobj.seek(7) 

pos = fileobj.tell() 

print('We are at character #%u.' % pos) 


You can also read any length from the file content during a given call. To do this pass an argument for read(). 
When read() is called with no argument it will read until the end of the file. If you pass an argument it will read that 
number of bytes or characters, depending on the mode (rb and r respectively): 


# reads the next 4 characters 

# starting at the current position 

next4 = fileobj.read(4) 

# what we got? 

print(next4) # ‘cucu' 

# where we are now? 

pos = fileobj.tell() 

print('We are at %u.' % pos) # We are at 11, as we was at 7, and read 4 chars. 


fileobj.close() 
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To demonstrate the difference between characters and bytes: 


with open('shoppinglist.txt', 'r') as fileobj: 
print(type(fileobj.read())) # <class ‘str'> 


with open('shoppinglist.txt', 'rb') as fileobj: 
print(type(fileobj.read())) # <class 'bytes'> 


Section 29.3: Read from stdin 
Python programs can read from unix pipelines. Here is a simple example how to read from stdin: 
import sys 


for line in sys.stdin: 
print(line) 


Be aware that sys.stdin is a stream. It means that the for-loop will only terminate when the stream has ended. 
You can now pipe the output of another program into your python program as follows: 
$ cat myfile | python myprogram.py 
In this example cat myfile can be any unix command that outputs to stdout. 
Alternatively, using the fileinout module can come in handy: 


import fileinput 
for line in fileinput.input(): 
process(line) 


Section 29.4: Using inputQ and raw_inputQ 
Python 2.x Version = 2.3 

raw_input will wait for the user to enter text and then return the result as a string. 
foo = raw_input("Put a message here that asks the user for input") 

In the above example foo will store whatever input the user provides. 

Python 3.x Version = 3.0 

input will wait for the user to enter text and then return the result as a string. 


foo = input("Put a message here that asks the user for input") 


In the above example foo will store whatever input the user provides. 


Section 29.5: Function to prompt user for a number 


def input_number(msg, err_msg=None) : 
while True: 
try: 
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return float(raw_input(msg) ) 
except ValueError: 
if err_msg is not None: 
print(err_msg) 


def input_number(msg, err_msg=None) : 
while True: 
try: 
return float(input(msg) ) 
except ValueError: 
if err_msg is not None: 
print(err_msg) 


And to use it: 


user_number = input_number( "input a number: "that's not a number!") 


Or, if you do not want an "error message": 


user_number = input_number("“input a number: ") 


Section 29.6: Printing a string without a newline at the end 
Python 2.x Version = 2.3 


In Python 2.x, to continue a line with print, end the print statement with a comma. It will automatically add a 
space. 


print "Hello, ", 
print "World!" 
# Hello, World! 


Python 3.x Version = 3.0 


In Python 3.x, the print function has an optional end parameter that is what it prints at the end of the given string. 


By default it's a newline character, so equivalent to this: 


print("Hello, ", end="\n") 
print("World!") 

# Hello, 

# World! 


But you could pass in other strings 


print("Hello, ", end="") 
print("World!") 
# Hello, World! 


print("Hello, ", end="<br>") 
print("World!") 

# Hello, <br>World! 
print("Hello, ", end="BREAK") 


print("World!") 
# Hello, BREAKWor1d! 


If you want more control over the output, you can use sys.stdout.write: 
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import sys 
sys.stdout.write("Hello, ") 


sys.stdout.write("World!") 
# Hello, World! 
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VIDEO: Python for Data 
Science and Machine 
Learning Bootcamp 


Learn how to use NumPy, Pandas, Seaborn, 
Matplotlib , Plotly, Scikit-Learn , Machine Learning, 
Tensorflow, and more! 


¥v Use Python for Data Science and Machine Learning 
v Use Spark for Big Data Analysis 

¥ Implement Machine Learning Algorithms 

¥ Learn to use NumPy for Numerical Data 

¥Y Learn to use Pandas for Data Analysis 

¥v Learn to use Matplotlib for Python Plotting 

v Learn to use Seaborn for statistical plots 

Vv Use Plotly for interactive dynamic visualizations 
Vv Use Scikit-Learn for Machine Learning Tasks 

v K-Means Clustering 

¥ Logistic Regression 

v Linear Regression 

¥ Random Forest and Decision Trees 

¥ Neural Networks 


v Support Vector Machines Ww re + re h T O ‘eo ‘eo v = 


Chapter 30: Files & Folders I/O 


Parameter Details 
filename the path to your file or, if the file is in the working directory, the filename of your file 


access_mode a string value that determines how the file is opened 
buffering an integer value used for optional line buffering 


When it comes to storing, reading, or communicating data, working with the files of an operating system is both 
necessary and easy with Python. Unlike other languages where file input and output requires complex reading and 
writing objects, Python simplifies the process only needing commands to open, read/write and close the file. This 
topic explains how Python can interface with files on the operating system. 


Section 30.1: File modes 
There are different modes you can open a file with, specified by the mode parameter. These include: 


e 'r' -reading mode. The default. It allows you only to read the file, not to modify it. When using this mode the 
file must exist. 


e ‘w' - writing mode. It will create a new file if it does not exist, otherwise will erase the file and allow you to 
write to it. 


e ‘a’ -append mode. It will write data to the end of the file. It does not erase the file, and the file must exist for 
this mode. 


e ‘rb’ - reading mode in binary. This is similar to r except that the reading is forced in binary mode. This is 
also a default choice. 


‘r+' - reading mode plus writing mode at the same time. This allows you to read and write into files at the 
same time without having to use r and w. 


‘rb+' - reading and writing mode in binary. The same as r+ except the data is in binary 


e ‘wb’ - writing mode in binary. The same as w except the data is in binary. 


‘w+' - writing and reading mode. The exact same as r+ but if the file does not exist, a new one is made. 
Otherwise, the file is overwritten. 


e ‘wb+' - writing and reading mode in binary mode. The same as w+ but the data is in binary. 


e ‘ab' - appending in binary mode. Similar to a except that the data is in binary. 


‘at’ - appending and reading mode. Similar to w+ as it will create a new file if the file does not exist. 
Otherwise, the file pointer is at the end of the file if it exists. 


‘ab+' - appending and reading mode in binary. The same as a+ except that the data is in binary. 


with open(filename, 'r') as f: 
f.read() 

with open(filename, ‘w') as f: 
f .write(filedata) 

with open(filename, ‘a') as f: 
f.write('\\n' + newdata) 


Read 4 JS KX 4 x Vv 
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Write x Vov 
Createsfile X xX Vv 
Erases file x x Vv 
Initial position Start Start Start Start End End 


Python 3 added a new mode for exclusive creation so that you will not accidentally truncate or overwrite and 
existing file. 


e 'x' - open for exclusive creation, will raise FileExistsError if the file already exists 

e 'xb' - open for exclusive creation writing mode in binary. The same as x except the data is in binary. 

'x+' - reading and writing mode. Similar to w+ as it will create a new file if the file does not exist. Otherwise, 
will raise FileExistsError. 

e 'xb+' - writing and reading mode. The exact same as x+ but the data is binary 


xX X+ 
Read x v 
Write Yedg 
Createsfile VY Vv 
Erases file x XxX 


Initial position Start Start 


Allow one to write your file open code in a more pythonic manner: 


Python 3.x Version = 3.3 


try: 
with open("fname", "r") as fout: 
# Work with your open file 
except FileExistsError: 
# Your error handling goes here 


In Python 2 you would have done something like 


Python 2.x Version = 2.0 
import os.path 
if os.path.isfile(fname) : 
with open("fname", "w") as fout: 
# Work with your open file 
else: 
# Your error handling goes here 


Section 30.2: Reading a file line-by-line 
The simplest way to iterate over a file line-by-line: 
with open('myfile.txt', 'r') as fp: 

for line in fp: 


print(line) 


readline() allows for more granular control over line-by-line iteration. The example below is equivalent to the one 
above: 


with open('myfile.txt', 'r') as fp: 


while True: 
cur_line = fp.readline() 
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# If the result is an empty string 

if cur_line == '': 
# We have reached the end of the file 
break 

print(cur_line) 


Using the for loop iterator and readline() together is considered bad practice. 
More commonly, the readlines() method is used to store an iterable collection of the file's lines: 
with open("myfile.txt", "r") as fp: 

lines = fp.readlines() 


for i in range(len(lines)): 
print("Line " + str(i) + 


+ line) 


This would print the following: 


Line 0: hello 


Line 1: world 


Section 30.3: Iterate files (recursively) 
To iterate all files, including in sub directories, use os.walk: 

import os 

for root, folders, files in os.walk(root_dir): 


for filename in files: 
print root, filename 


root_dir can be "." to start from current directory, or any other path to start from. 
Python 3.x Version = 3.5 


If you also wish to get information about the file, you may use the more efficient method os.scandir like so: 


for entry in os.scandir (path): 
if not entry.name.startswith('.') and entry.is_file(): 
print(entry.name) 


Section 30.4: Getting the full contents of a file 


The preferred method of file i/o is to use the with keyword. This will ensure the file handle is closed once the 
reading or writing has been completed. 


with open('myfile.txt') as in_file: 
content = in_file.read() 


print(content) 


or, to handle closing the file manually, you can forgo with and simply call close yourself: 


in_file = open('myfile.txt', ‘r') 
content = in_file.read() 
print(content) 
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in_file.close() 


Keep in mind that without using a with statement, you might accidentally keep the file open in case an unexpected 
exception arises like so: 


in_file = open('myfile.txt', 'r') 
raise Exception("oops") 
in_file.close() # This will never be called 


Section 30.5: Writing to a file 


with open('myfile.txt', ‘w') as f: 
-write("Line 1") 
-write("Line 2") 
-write("Line 3") 
-write("Line 4") 


>h aha 


If you open myfile. txt, you will see that its contents are: 


Line 1Line 2Line 3Line 4 


Python doesn't automatically add line breaks, you need to do that manually: 


with open('myfile.txt', ‘w') as f: 
-write("Line 1\n") 
-write("Line 2\n") 
-write("Line 3\n") 
-write("Line 4\n") 


> haa 


Line 1 
Line 2 
Line 3 
Line 4 


Do not use os. linesep as a line terminator when writing files opened in text mode (the default); use \n instead. 


If you want to specify an encoding, you simply add the encoding parameter to the open function: 


with open('my_file.txt', 'w', encoding='utf-8') as f: 


f.write('utf-8 text’) 


It is also possible to use the print statement to write to a file. The mechanics are different in Python 2 vs Python 3, 
but the concept is the same in that you can take the output that would have gone to the screen and send it to a file 
instead. 


Python 3.x Version = 3.0 


with open('fred.txt', 'w') as outfile: 
s = "I'm Not Dead Yet!" 
print(s) # writes to stdout 
print(s, file = outfile) # writes to outfile 


#Note: it is possible to specify the file parameter AND write to the screen 
#by making sure file ends up with a None value either directly or via a variable 
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myfile = None 
print(s, file 
print(s, file 


myfile) # writes to stdout 
None) # writes to stdout 


In Python 2 you would have done something like 


Python 2.x Version = 2.0 


outfile = open('fred.txt', ‘w') 


s = "I'm Not Dead Yet!" 
print s  # writes to stdout 
print >> outfile, s  # writes to outfile 


Unlike using the write function, the print function does automatically add line breaks. 


Section 30.6: Check whether a file or path exists 
Employ the EAFP coding style and try to open it. 


import errno 


try: 
with open(path) as f: 
# File exists 
except IOError as e: 
# Raise the exception if it is not ENOENT (No such file or directory) 
if e.errno != errno.ENOENT: 
raise 
# No such file or directory 


This will also avoid race-conditions if another process deleted the file between the check and when it is used. This 
race condition could happen in the following cases: 


e Using the os module: 


import os 
os.path.isfile('/path/to/some/file.txt' ) 


Python 3.x Version = 3.4 


e Using pathlib: 


import pathlib 
path = pathlib.Path('/path/to/some/file.txt') 
if path.is_file(): 


To check whether a given path exists or not, you can follow the above EAFP procedure, or explicitly check the path: 


import os 
path = "/home/myFiles/directory1" 


if os.path.exists(path) : 
## Do stuff 
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